|A Systematic Replication Study of Interleaved Mathematics Practice
|Research Grants Focused on Systematic Replication [Program Details]
|5 years (08/01/2022 - 07/31/2027)
Co-Principal Investigators: Davenport, Jodi; Rohrer, Doug; Heffernan, Cristina; Heffernan III, Neil T.; Shaw, Stacy
Purpose: The purpose of this project is to conduct a systematic replication study of a highly promising mathematics learning intervention, interleaved practice, in 7th grade classrooms. Though psychologists have long known that interleaving and spacing improves long-term learning, the practice problems in most mathematics curricula are arranged so that the majority of problems relating to the same skill or concept are blocked together. With the interleaved practice intervention, some of the practice problems are rearranged so that 1) problems of different kinds are mixed together, which improves learning, and 2) problems of the same kind are distributed across multiple assignments, which improves retention. Numerous studies in the laboratory and classroom have demonstrated that merely rearranging practice problems so that the students receive a higher dose of interleaved practice can dramatically boost scores on researcher-developed measures of learning. The systematic replication study will determine whether this promising intervention can improve scores on externally-developed outcome measures and whether these intervention can scale to a widely-used online intervention that currently reaches tens of thousands of students in diverse settings.
Project Activities: The researchers will conduct the study with a representative sample of students in the U.S. east coast with a diverse representation of mathematical proficiency, socioeconomic background, gender, and ethnicity. The team will replicate and extend prior work by including educationally relevant outcome measures, including students at a variety of proficiency levels, testing long term retention, and leveraging the ASSISTments digital mathematics learning platform. The study will involve a cluster-randomized trial, randomizing classrooms within teachers to implement interleaved practice problems or blocked practice problems in the grade 7 mathematics school year. Seventy grade 7 teachers across the U.S. east coast will participate, with approximately 189 classrooms, and 3,780 students. Student outcomes will be assessed at the end of the school year on proximal and distal tests as well as at the beginning of the 8th grade year. The study will collect fidelity and implementation data and will conduct a cost analysis.
Products: The project team will systematically extend past research on interleaving, producing a firmer understanding of the extent to which interleaving replicates in a larger, more heterogenous sample, as well as how and for whom interleaving benefits learning. As interleaved practice appears to be highly effective yet scarcely used intervention that can be inexpensively implemented in nearly any mathematics course, the findings from the replication will be of interest to researchers, practitioners and policymakers. Dissemination products will include journal articles as well as practitioner-focused webinars, blogs, articles, presentations, and a final publicly accessible data set.
Setting: The setting includes 13 U.S. east coast states that have adopted the Common Core State Standards.
Sample: The sample is representative of the population with balanced representation along free/reduced lunch status, underrepresented in STEM minority status, gender, and baseline mathematics achievement.
Intervention: Interleaved practice can be added to any curriculum by rearranging a portion of the practice problems. The intervention is applied by rearranging existing practice problems along the following two criteria: (1) The practice problems in a course are arranged so that most problems follow a problem relating to a different skill or concept so that problems of different kinds are interleaved within an assignment, and (2) Most of the practice problems relating to a particular skill or concept are distributed across many assignments. The systematic replication will apply the interleaved strategy by rearranging problems from existing Grade 7 math curricula that are freely available within ASSISTments, a widely-used digital learning platform.
Research Design and Methods: The researchers will conduct a blocked, randomized controlled trial, with random assignment occurring at the classroom-level. The teacher will serve as the block, and the impact on student mathematics achievement will be measured both immediately and after a six-month delay. The study will consist of a year-long implementation to take place across two-cohorts (in the second and third years of the project).
Control Condition: The counterfactual will consist of business-as-usual (BAU) control that uses existing math curricula freely available within the ASSISTments platform.
Key Measures: Student outcome measures will include a researcher developed proximal measure of student mathematics achievement, the Grade 8 Mathematics Readiness test, and the state standardized mathematics tests for 7th grade.
Data Analytic Strategy: The research team will use hierarchical linear modeling, regressing the student outcome on condition assignment and controlling for student, classroom, and teacher/school covariates, and including random effects for classroom (level of assignment) and teachers/school (block). The analysis will be repeated for each outcome and post hoc adjustments will be applied to account for multiple comparisons.
Cost Analysis: The research team will gather costs using the "ingredients method" and will include all expenditures on personnel, facilities, equipment, materials, and training. Upon completion of the cost analysis and impact analysis, the researchers will compute cost-effectiveness ratios to determine the amount of additional money spent on the treatment (as compared to the control group) in relation to the amount of additional achievement of the treatment group, where achievement is measured in standard deviation units.
Related IES Projects: An Efficacy Study of Interleaved Mathematics Practice (R305A160263); Interleaved Mathematics Practice (R305A110517)