Schools were randomly assigned to one of four different curricula, using a stratified procedure that helped allocate similar numbers and types of schools to each curriculum. All first-grade classrooms in participating schools were included in the study. When compared to the national average, participating schools had a higher percentage of minority students and students eligible for free/reduced-price meals.

The baseline sample consisted of four districts, 40 schools, 134 teachers, and 1,525 first-grade students. The analysis sample consisted of four districts, 39 schools, 131 teachers, and 1,309 first-grade students: 11 schools with 36 teachers and 359 students used Scott Foresman–Addison Wesley Elementary Mathematics (the intervention), 10 schools with 33 teachers and 332 students used Investigations in Number, Data, and Space (comparison 1), 9 schools with 31 teachers and 314 students used Math Expressions (comparison 2), and 9 schools with 31 teachers and 304 students used Saxon Math (comparison 3).

One school with 3 teachers and 32 students assigned to Math Expressions withdrew from the study and did not permit posttesting of students. Because this represents differential attrition of more than 5 percentage points for the comparison of Scott Foresman–Addison Wesley Elementary Mathematics and Math Expressions, this particular comparison is rated as meeting evidence standards with reservations.

The authors compared the baseline characteristics of the students in the analysis sample on seven characteristics, including the baseline assessment score. Statistical tests conducted on those characteristics for the analysis sample indicated no differences across the four groups. Subgroup findings based on school and classroom characteristics, including baseline fall math achievement and free/reduced-price meal eligibility, are provided in Appendix A4.

Comparison 1 students used Investigations in Number, Data, and Space as their core math curriculum. The curriculum is published by Pearson Scott Foresman. It uses a student-centered approach that encourages reasoning and understanding and draws on constructivist learning theory. The lessons focus on understanding, rather than on "correct answers," and build on students' knowledge and understanding. Students are engaged in thematic units of three to eight weeks in which they first investigate and then discuss and reason about problems and strategies. Students frequently create their own representations. Some 80% of teachers reported completing at least 80% of the curriculum (not significantly different from the other three curricula, p-value = 0.24).

Comparison 2 students used Math Expressions as their core math curriculum. Math Expressions is published by Houghton Mifflin Harcourt and uses a blend of student-centered and teacher-directed instructional approaches. Students using the curriculum question and discuss mathematics but are explicitly taught effective procedures. There is an emphasis on using multiple specified objects, drawings, and language to represent concepts, and an emphasis on learning through the use of real-world situations. Students are expected to explain and justify their solutions. Some 89% of teachers reported completing at least 80% of the curriculum (not significantly different from the other three curricula, p-value = 0.24).

Comparison 3 students used Saxon Math as their core math curriculum. Saxon Math is published by Houghton Mifflin Harcourt and uses a teacher-directed approach that offers a script for teachers to follow in each lesson. The curriculum blends teacher-directed instruction of new material with daily distributed practice of previously learned concepts and procedures. The teacher introduces concepts or efficient strategies for solving problems. Students observe and then receive guided practice, followed by distributed practice. Students hear the correct answers and are explicitly taught procedures and strategies. Frequent monitoring of student achievement is built into the program. Daily routines are extensive and emphasize practice of number concepts and procedures and use of representations. Some 97% of teachers reported completing at least 80% of the curriculum (not significantly different from the other three curricula, p-value = 0.24).

Intervention: All teachers were provided one day of initial training in the summer before the school year began. More than 90% of teachers reported feeling adequately or very well prepared to use the intervention after the initial training. Follow-up training was offered about every four to six weeks throughout the school year. Follow-up sessions were typically three to four hours long and held after school.

Comparison 1: Teachers assigned to Investigations in Number, Data, and Space were provided one day of initial training in the summer before the school year began. More than 90% of teachers reported feeling adequately or very well prepared to use the curriculum after the initial training. Follow-up training was offered about every four to six weeks throughout the school year. Follow-up sessions were typically three to four hours long and held after school.

Comparison 2: Teachers assigned to Math Expressions were provided two days of initial training in the summer before the school year began. Some 54% of teachers reported feeling adequately or very well prepared to use the curriculum after the initial training. Two follow-up trainings were offered during the school year. Follow-up sessions typically consisted of classroom observations followed by short feedback sessions with teachers.

Comparison 3: Teachers assigned to Saxon Math were provided one day of initial training in the summer before the school year began. More than 90% of teachers reported feeling adequately or very well prepared to use the curriculum after the initial training. One follow-up training session was offered during the school year and tailored to meet each district's needs.

Third- and 5th-grade teachers were randomly assigned to the intervention or comparison condition. The baseline sample included 39 teachers (20 treatment and 19 comparison) and 915 students (468 treatment and 447 comparison). Twenty-three teachers taught 3rd grade (13 treatment and 10 comparison), and 16 taught 5th grade (7 treatment and 9 control). No teachers left the study, and student attrition was low. Between 837 and 863 students were posttested on the TerraNova Math Computation and Math Total assessments, respectively.² In general, participating schools had a higher percentage of Asian students and students with higher ability levels than the national average. Participating schools had a lower percentage of Hispanic and African-American students, special education students, and students eligible for free/reduced-price meals than the national average.

Comparison students used three different math curricula. Students in two schools used a chapter-based, comprehensive basal program; students in a third school used a basal math program; and students in a fourth school used a school-created math program based on a number of different math materials from various resources. The comparison curricula generally covered the same content as Scott Foresman–Addison Wesley Elementary Mathematics. Teachers covered 80% (SD = 9.5%) of the curricula.

Teachers received three hours of initial training prior to implementing Scott Foresman–Addison Wesley Elementary Mathematics in their classes. At the initial training session, the trainer described the key components of Scott Foresman–Addison Wesley Elementary Mathematics, reviewed the Teacher's Edition and available ancillary resources, offered examples of when to use certain materials, provided an overview of the math technology available, and modeled a math lesson. The training focused on the components most vital to the program and those that were required for full implementation.

Two follow-up sessions were offered during the school year. The first session was offered four to eight weeks into the school year and lasted two hours. The session was informal and allowed teachers to discuss and ask questions about issues encountered while implementing the program. A second follow-up session was provided to one school in March (the other three schools were offered the second follow-up session but chose not to receive it). The second follow-up addressed pacing issues and further covered the technology available with the program.

Second- and 4th-grade teachers were randomly assigned to the intervention using Scott Foresman–Addison Wesley Elementary Mathematics. The baseline sample included 35 teachers (18 treatment and 17 comparison) and 742 students (389 treatment and 353 comparison). Of the 35 study teachers, 19 taught 2nd grade (10 treatment and 9 comparison) and 16 taught 4th grade (8 treatment and 8 comparison). The analysis sample included 35 teachers (18 treatment and 17 comparison) and 533 (290 treatment and 243 comparison) to 645 (352 treatment and 293 comparison) students in the TerraNova Math Computation and Math Total analyses, respectively.¹ For both assessments, the differential attrition exceeded 5 percentage points; therefore, this study is rated as meeting evidence standards with reservations. Some 37% of participating students were minorities. At two of the six participating schools, more than 90% of students were eligible for free/reduced-price meals; the percentage of students eligible for free/reduced-price meals at the other four schools was similar to the national average of 37%.

Students used five different comprehensive math curricula that used basal or investigative approaches. The comparison curricula covered the same content as Scott Foresman–Addison Wesley Elementary Mathematics. Teachers covered 75% (SD = 18.2%) of the curricula.

Teachers in the intervention classrooms met with a Scott Foresman–Addison Wesley Elementary Mathematics professional trainer for approximately four hours prior to implementing the curriculum in their classes. In the initial training session, the trainer described the key components of the curriculum, reviewed the materials provided, and offered examples of when to use certain materials.

Two follow-up sessions, approximately two hours each, were offered. The first session occurred 4 to 8 weeks after teachers began implementation. A second session occurred 10 to 18 weeks after implementation and was provided by the Scott Foresman–Addison Wesley Elementary Mathematics trainer and one of the curriculum authors. This second session focused on the curriculum's philosophy, lesson modeling, and how teachers could use Scott Foresman–Addison Wesley Elementary Mathematics to help students understand mathematics. The second session was provided to five of the six schools.

Mixed effects: Evidence of inconsistent effects as demonstrated through either of the following criteria.

• Criterion 1: At least one study showing a statistically significant or substantively important positive effect, and at least one study showing a statistically significant or substantively important negative effect, but no more such studies than the number showing a statistically significant or substantively important positive effect.

  Not met. No studies showed a statistically significant or substantively important positive effect.

OR

• Criterion 2: At least one study showing a statistically significant or substantively important effect, and more studies showing an indeterminate effect than showing a statistically significant or substantively important effect.

  Met. One study showed statistically significant negative effects, and two studies showed indeterminate effects.

Positive effects: Strong evidence of a positive effect with no overriding contrary evidence.

• Criterion 1: Two or more studies showing statistically significant positive effects, at least one of which met WWC evidence standards for a strong design.

  Not met. No studies showed a statistically significant or substantively important positive effect.

  AND

• Criterion 2: No studies showing statistically significant or substantively important negative effects.

  Not met. One study showed statistically significant negative effects.

Potentially positive effects: Evidence of a positive effect with no overriding contrary evidence.

• Criterion 1: At least one study showing a statistically significant or substantively important positive effect.

  Not met. No studies showed a statistically significant or substantively important positive effect.

  AND

• Criterion 2: No studies showing a statistically significant or substantively important negative effect and fewer or the same number of studies showing indeterminate effects than showing statistically significant or substantively important positive effects.

  Not met. One study showed statistically significant or substantively important negative effects, and two studies showed indeterminate effects.