|
|
Four studies reviewed by the WWC investigated the effects of the Houghton Mifflin Mathematics program. Two studies were quasi-experimental designs that met WWC evidence standards with reservations. The two remaining studies did not meet WWC evidence screens.
Johnson and Hall (2003) included 160 intervention schools in eight California districts using Houghton Mifflin Mathematics (2002 edition) in grades 2-5 and 137 comparison schools in eight different districts using non-Houghton Mifflin programs. The intervention schools had completed their first year of implementing Houghton Mifflin Mathematics. The comparison school districts were matched to the intervention districts based on prior mathematics achievement scores on California's Stanford 9 test, student demographic characteristics, and district sizes. Selection of comparison school districts relied on data from the California Department of Education, the Quality Education Database, and the American Institutes for Research. Statistical analyses of the math scores for the intervention districts and the comparison districts collected during the baseline year (2000-01) showed that, prior to the introduction of Houghton Mifflin Mathematics, there were no statistically significant differences between the two groups of schools at any grade level.
The EDSTAR, Inc. (2004) study was conducted in 519 schools from 32 school districts (16 district pairs) in California, Illinois, Missouri, Wisconsin, New Jersey, New York, and South Carolina. The intervention group included 308 schools from 16 districts using Houghton Mifflin Mathematics (2002 edition) for the first time during the 2002-03 school year. The comparison group included 211 schools from 16 different districts using reform, traditional, or balanced math programs. Math programs were classified as reform if they placed more emphasis on conceptual understanding than on traditional computation skills. Traditional programs emphasized computational skills, while balanced programs integrated conceptual understanding with traditional computational skills.
In each of the 16 district pairs in the EDSTAR, Inc. (2004) study, the intervention and comparison districts were matched based on prior mathematics achievement scores for the baseline year (2001-02), student demographics, district size, and average school size. No statistically significant differences in math achievement scores for the baseline year were found between the intervention and comparison groups. The WWC determined that having one district in the intervention group and a separate district in the comparison group confounded the intervention effect with the district. 2 The intervention effect could not be disentangled from other district characteristics without limiting the study to states that had multiple districts in the intervention and comparison groups. The authors provided additional information that enabled the district data to be separated by state. The WWC analyses are based on the reduced sample of three states, eight district pairs (16 districts), and 212 schools. The three states in the reduced sample were California, South Carolina, and New Jersey. California had two district pairs (four districts) and 68 schools. South Carolina had four district pairs (eight districts) and 128 schools. And New Jersey had two district pairs (four districts) and 16 schools. In the reduced sample all of the comparison districts within a state used the same type of math program (reform, traditional, or balanced).
The WWC categorizes the extent of evidence in each domain as small or medium to large (see the What Works Clearinghouse Extent of Evidence Categorization Scheme). The extent of evidence takes into account the number of studies and the total sample size across the studies that met WWC evidence standards with or without reservations. 3
The WWC considers the extent of evidence for Houghton Mifflin Mathematics to be medium to large for math achievement.
|Institute of Education Sciences
