Design-Based Research within the Constraints of Practice: AlgebraByExample
Booth, Julie L.; Cooper, Laura A.; Donovan, M. Suzanne; Huyghe, Alexandra; Koedinger, Kenneth R.; Paré-Blagoev, E. Juliana (2015). Journal of Education for Students Placed at Risk, v20 n1-2 p79-100. Retrieved from: https://eric.ed.gov/?id=EJ1056824
-
examining380Students, grades8-12
Single Study Review
Review Details
Reviewed: June 2026
- Single Study Review (findings for AlgebraByExample - Booth et al. (2015))
- Randomized Controlled Trial
- Meets WWC standards with reservations because at least one supplemental finding is rated meets WWC standards without reservations but the study does not have main findings.
This review may not reflect the full body of research evidence for this intervention.
Evidence Tier rating based solely on this study. This intervention may achieve a higher tier when combined with the full body of evidence.
Findings
|
Outcome measure |
Comparison | Period | Sample |
Intervention mean |
Comparison mean |
Significant? |
Improvement index |
||
|---|---|---|---|---|---|---|---|---|---|
| Show Supplemental Findings | |||||||||
|
Items from five standardized state tests (Booth et al., 2015) |
AlgebraByExample - Booth et al. (2015) vs. Business as usual |
0 Days |
Full sample;
|
0.49 |
0.48 |
Yes |
|
||
|
Conceptual Algebra Knowledge (Booth et al. 2015) |
AlgebraByExample - Booth et al. (2015) vs. Business as usual |
0 Days |
Full sample;
|
0.51 |
0.49 |
Yes |
|
||
|
Procedural Algebra Knowledge (Booth et al. 2015) |
AlgebraByExample - Booth et al. (2015) vs. Business as usual |
0 Days |
Full sample;
|
0.25 |
0.25 |
Yes |
|
||
Evidence Tier rating based solely on this study. This intervention may achieve a higher tier when combined with the full body of evidence.
Sample Characteristics
Characteristics of study sample as reported by study author.
-
Female: 53%
Male: 47% -
Suburban, Urban
-
Race Asian 7% Black 39% Other or unknown 18% Two or more races 6% White 30% -
Ethnicity Hispanic 18% Not Hispanic or Latino 82% -
Eligible for Free and Reduced Price Lunch Other or unknown 100%
Study Details
Setting
This study took place in five school districts across five states that are part of the Minority Student Achievement Network (MSAN). The authors did not report the regions or states in which the study took place. The districts were predominantly suburban and small urban. The study was conducted in nonhonors Algebra I classrooms.
Study sample
The study included 380 students in 25 classrooms (189 used AlgebraByExample and 191 did not). Eleven teachers participated in the study. The study included 13 grade 8 classrooms and 12 high school classrooms. It is likely that most students in the high school classrooms were in grade 9.
Intervention Group
Students in the intervention group received AlgebraByExample, a supplemental set of 42 Algebra I assignments designed to improve conceptual and procedural understanding through the use of worked examples and self-explanation prompts. Each assignment included 6–8 algebra problems aligned to topics typically covered in Algebra I. In the intervention condition, some problems were replaced with worked examples that provided step-by-step solutions. Students were asked to explain why the solutions were correct or incorrect, with assignments including both correct and incorrect examples to highlight common misconceptions. Worked examples were alternated with practice items within each assignment. The assignments were compiled into workbooks and distributed at the beginning of the school year. Instruction and assessments occurred as part of typical classroom activities. The intervention was implemented within existing coursework and did not replace the core Algebra I curriculum. Although students received workbooks containing 42 assignments, teachers did not necessarily use all assignments. Across classrooms, teachers used between 15 and 40 assignments, with an average of about 26 assignments.
Comparison Group
The comparison group used the same curriculum-based instruction in Algebra I as the intervention group, with 42 in-class assignments designed in parallel to those used in the intervention group. Each comparison assignment contained the same 6-8 items as the corresponding intervention assignment, but without the worked examples and self-explanation prompts.
Support for implementation
Researchers collaborated with district math coordinators and participating teachers prior to the study to develop and refine the AlgebraByExample assignments and ensure alignment with district curricula. The study does not describe any ongoing implementation support during the study period.
An indicator of the effect of the intervention, the improvement index can be interpreted as the expected change in percentile rank for an average comparison group student if that student had received the intervention.
For more, please see the WWC Glossary entry for improvement index.
An outcome is the knowledge, skills, and attitudes that are attained as a result of an activity. An outcome measures is an instrument, device, or method that provides data on the outcome.
A finding that is included in the effectiveness rating. Excluded findings may include subgroups and subscales.
The sample on which the analysis was conducted.
The group to which the intervention group is compared, which may include a different intervention, business as usual, or no services.
The timing of the post-intervention outcome measure.
The number of students included in the analysis.
The mean score of students in the intervention group.
The mean score of students in the comparison group.
The WWC considers a finding to be statistically significant if the likelihood that the finding is due to chance alone, rather than a real difference, is less than five percent.
The WWC reviews studies for WWC products, Department of Education grant competitions, and IES performance measures.
The name and version of the document used to guide the review of the study.
The version of the WWC design standards used to guide the review of the study.
The result of the WWC assessment of the study. The rating is based on the strength of evidence of the effectiveness of the intervention. Studies are given a rating of Meets WWC Design Standards without Reservations, Meets WWC Design Standards with Reservations, or >Does Not Meet WWC Design Standards.
A related publication that was reviewed alongside the main study of interest.
Study findings for this report.
Based on the direction, magnitude, statistical significance, and sample size of the findings within a domain, the WWC assigns effectiveness ratings as one of the following: Tier 1 (strong evidence), Tier 2 (moderate evidence), Tier 3 (promising evidence), uncertain effects, and negative effects. For more detail, please see the WWC Handbook.
The WWC may review studies for multiple purposes, including different reports and re-reviews using updated standards. Each WWC review of this study is listed in the dropdown. Details on any review may be accessed by making a selection from the drop down list.
Tier 1 Strong indicates strong evidence of effectiveness,
Tier 2 Moderate indicates moderate evidence of effectiveness, and
Tier 3 Promising indicates promising evidence of effectiveness,
as defined in the
non-regulatory guidance for ESSA
and the regulations for ED discretionary grants (EDGAR Part 77).