Appendix A1 Extent of evidence
| Intervention name | Number of studies | Sample size (schools/students) | Extent of evidence1 |
|---|---|---|---|
| Cognitive Tutor | 2 | 9/781 | Medium to large |
| Connected Mathematics Project (CMP) | 3 | 100/14,696 | Medium to large |
| I CAN Learn® Pre-Algebra and Algebra | 6 | 729/16,656 | Medium to large |
| Saxon Middle School Math | 6 | 101/3,399 | Medium to large |
| The Expert Mathematician | 1 | 1/70 | Small |
| Transition Mathematics | 3 | 49/972 | Medium to large |
| UCSMP Algebra | 2 | 4/225 | Medium to large 2 |
|
nr = not reported 1 A rating of "medium to large" requires at least two studies and two schools across studies in one domain and a total sample size across studies of at least 350 students or 14 classrooms. Otherwise, the rating is "small."2 The extent of evidence for UCSMP Algebra is considered to be medium to large because, across studies, 14 classrooms were included at the time of analysis. |
|||
Appendix A2 Targeted population
| Intervention name | Targeted students (grade levels) | Students in studies reviewed (grade levels)1 |
|---|---|---|
| Cognitive Tutor | 7–12 | 9 |
| Connected Mathematics Project (CMP) | 6–8 | 6–8 |
| I CAN Learn® Pre-Algebra and Algebra | 6–12 | 8–9 |
| Saxon Middle School Math | 6–9 | 6–9 |
| The Expert Mathematician | 6–9 | 8 |
| Transition Mathematics | 7–12 | 7–9 |
| UCSMP Algebra | 7–10 | 8–9 |
|
Note: This table compares targeted grade levels and the grade levels in the studies reviewed by the WWC. Grade levels are related to student age and may affect outcomes due to differences in the students' developmental stages as well as differences in school size and organization. 1 Some of the studies reviewed included students in grades 10 or above, but the findings for those students were not reviewed because those grades were outside the scope of this review. |
||
Appendix A3 Summary of statistically significant1 or substantively important2 positive outcomes
| Intervention name | Math achievement | |
|---|---|---|
| Statistically significant positive findings3 | Math achievement across outcomes | |
| Cognitive Tutor | ||
| Morgan & Ritter, 2002 | Math achievement grades (end of first and second semesters) | Statistically significant, Substantively important |
| Schneyderman, 2001 | ns | ns, nsi |
| Connected Mathematics Project (CMP) | ||
| Ridgway, Zawojewski, Hoover, & Lambdin, 2002 | ns | ns, nsi |
| Riordan & Noyce, 2001 | Massachusetts Comprehensive Assessment System (MCAS)—math scores | ns4 |
| Schneider, 2000 | ns | ns, nsi |
| I CAN Learn® Pre-Algebra and Algebra | ||
| Kirby, 2006, October | Louisiana Educational Assessment Program (LEAP) Grade 8 Mathematics Exam | Statistically significant, Substantively important |
| Kerstyn, 2001, Algebra 1 | ns | ns, nsi |
| Kerstyn, 2001, Algebra 1 Honors | ns | ns, nsi |
| Kerstyn, 2001, MJ-3 pre-algebra | ns | ns, nsi |
| Kerstyn, 2001, MJ-3 Advanced | ns | ns, nsi |
| Kerstyn, 2002, October, Algebra 1 | ns | ns, nsi |
| Kerstyn, 2002, October, Algebra 1 Honors | ns | ns, nsi |
| Kerstyn, 2002, October, MJ-3 pre-algebra | FCAT mathematics | Statistically significant, nsi |
| Kerstyn, 2002, October, MJ-3 Advanced | ns | ns, nsi |
| Kirby, 2004, September | General Mathematics CST | Statistically significant, Substantively important |
| Kirby, 2004a, November | Georgia Criterion-Referenced Competency Test (GCRCT) Math Test | Statistically significant, Substantively important |
| Kirby, 2005, January | Algebra 1 EOC test | Statistically significant, Substantively important |
| Saxon Middle School Math | ||
| Williams, 1986 | End-of-course math test | Statistically significant, Substantively important |
| Peters, 1992 | ns | ns, nsi |
| Crawford & Raia, 1986 | The California Achievement Test (CAT) | Statistically significant, Substantively important |
| Resendez, Fahmy, & Manley, 2005 | The Texas Assessment of Academic Skills (TAAS)—TLI score; The Texas Assessment of Knowledge and Skills (TAKS) | Statistically significant, nsi |
| Resendez & Manley, 2005 | ns | ns4 |
| Roberts, 1994 | ns | ns, nsi |
| The Expert Mathematician | ||
| Baker, 1997 | ns | ns, Substantively important |
| Transition Mathematics | ||
| Baker, 1997 | ns | ns, Substantively important negative effect |
| Hedges et al., 1986 | Geometry Readiness | ns, nsi |
| Thompson et al., 2005 | ns | ns, nsi |
| UCSMP Algebra | ||
| Peters, 1992 | ns | ns, nsi |
| Thompson et al., 2006 | Algebra Readiness; Problem Solving and Understanding | ns, Substantively important |
|
na = not studied 2 For rating purposes, the WWC considers the statistical significance of the findings and the magnitude of the effect, also called the effect size. An average effect size is the sum of all the effect sizes of the student outcomes in a study in a single domain divided by the number of those outcomes. The WWC considers an average effect size across all student outcomes in one study in a given domain to be substantively important if it is equal to or greater than 0.25. 3 No studies showed statistically significant negative effects on math achievement. 4 Student-level effect size could not be computed for this study; whether or not the magnitude of the effect is substantively important is unknown. However, the statistical significance for this study is comparable to other studies and is included in the intervention rating. For further details, see Technical Details of WWC-Conducted Computations. |
||
|Institute of Education Sciences
