Effectiveness

Findings

The WWC review of interventions for Middle School Math addresses student outcomes in the math achievement domain. The findings below present the authors’ estimates and WWC-calculated estimates of the size and the statistical significance of the effects of Saxon Math on students.¹²

Math achievement

Resendez and Azin (2006) reported positive but not statistically significant effects of Saxon Math on math achievement based on TerraNova Math Total and TerraNova Math Computation Total test scores. The effect sizes calculated by the WWC were not large enough to be considered substantively important according to WWC criteria (an effect size of at least 0.25).¹³

Peters (1992) reported a positive but not statistically significant effect of Saxon Math on math achievement based on Orleans-Hanna Prognosis Test test scores. The effect size was not large enough to be considered substantively important by WWC criteria (an effect size of at least 0.25).

Crawford and Raia (1986) reported a positive and statistically significant effect of Saxon Math on math achievement based on California Achievement Test (CAT) Total Math test scores. The statistical significance of this effect was not confirmed by WWC analysis.¹⁴ However, the effect size was large enough to be considered substantively important according to WWC criteria (an effect size of at least 0.25).

Resendez, Fahmy, and Manley (2005, Cohort A) reported a positive and statistically significant effect of Saxon Math for the grade 8 cohort on math achievement based on the Texas Assessment of Academic Skills (TAAS) math scores. After accounting for the use of different levels of assignment and analysis, the WWC determined this finding was not statistically significant or large enough to be considered substantively important by WWC criteria (an effect size of at least 0.25).

Resendez, Fahmy, and Manley (2005, Cohort F) reported a positive and statistically significant effect of Saxon Math for the grade 6 cohort on math achievement based on the Texas Assessment of Knowledge and Skills (TAKS) math test. After accounting for the use of different levels of assignment and analysis, the WWC determined that this finding was not statistically significant, but the effect size was large enough to be considered substantively important according to WWC criteria (an effect size of at least 0.25).

In sum, in the math achievement domain, the WWC reviewed findings from five studies. Two of these studies showed substantively important positive effects, and the remaining three studies showed indeterminate effects. One study used a strong design.

Rating of effectiveness

The WWC rates the effects of an intervention in a given outcome domain as positive, potentially positive, mixed, no discernible effects, potentially negative, or negative. The rating of effectiveness takes into account four factors: the quality of the research design, the statistical significance of the findings, the size of the difference between participants in the intervention and the comparison conditions, and the consistency in findings across studies (see the WWC Procedures and Standards Handbook, Appendix E).