Effectiveness

Findings

The WWC review of elementary school mathematics curriculum-based interventions addresses student outcomes in mathematics achievement.

Mathematics achievement. Johnson and Hall (2003) reported significant, positive effects for Houghton Mifflin Mathematics on overall mathematics achievement for grades 2-5. Because the authors presented average school-level math achievement gains and pretest scores but no posttest scores, the WWC requested school-level average posttest scores from the authors. ³ Using the school-level data provided by the authors, and after accounting for clustering,⁴ the WWC determined that the effect of Houghton Mifflin Mathematics on math achievement was neither statistically significant nor substantively important, according to WWC criteria. Thus the WWC categorized the effect of Houghton Mifflin Mathematics on mathematics achievement as indeterminate.

The EDSTAR, Inc. (2004) study used a series of comparisons between a single treatment district and a single comparison district. This analysis does not allow the effect of the intervention to be disentangled from the effect of other district characteristics. As a result, the WWC requested that the authors aggregate the data in each of the three states that included multiple treatment districts and multiple comparison districts. ⁵ This reanalysis eliminated the confound between intervention effects and district effects. Using school-level data provided to the WWC by the authors for the three states that had multiple districts in the intervention and comparison groups and accounting for clustering,⁴ the WWC determined that the effect of Houghton Mifflin Mathematics on mathematics achievement was neither statistically significant nor substantively important, according to WWC criteria. Thus the WWC categorized the effect of Houghton Mifflin Mathematics on mathematics achievement as indeterminate.

Rating of effectiveness

The WWC rates the effects of an intervention in a given outcome domain as positive, potentially positive, mixed, no discernible effects, potentially negative, or negative. The rating of effectiveness takes into account four factors: the quality of the research design, the statistical significance of the findings (as calculated by the WWC⁴ ), the size of the difference between participants in the intervention condition and the comparison condition, and the consistency in findings across studies (see the WWC Intervention Rating Scheme). The WWC found Houghton Miffin to have no discernible effects for mathematics achievement.

Top