Achievement Effects of Four Early Elementary School Math Curricula: Findings for First and Second Graders. NCEE 2011-4001
Agodini, Roberto; Harris, Barbara; Thomas, Melissa; Murphy, Robert; Gallagher, Lawrence (2010). National Center for Education Evaluation and Regional Assistance. Retrieved from: https://eric.ed.gov/?id=ED512551
-
examining8,060Students, grades1-2
Math Expressions Intervention Report - Primary Mathematics
Review Details
Reviewed: March 2021
- Randomized Controlled Trial
- Meets WWC standards without reservations because it is a cluster randomized controlled trial with low cluster-level attrition and individual-level non-response.
This review may not reflect the full body of research evidence for this intervention.
Evidence Tier rating based solely on this study. This intervention may achieve a higher tier when combined with the full body of evidence.
Please see the WWC summary of evidence for Math Expressions.
Findings
|
Outcome measure |
Comparison | Period | Sample |
Intervention mean |
Comparison mean |
Significant? |
Improvement index |
Evidence tier |
|
|---|---|---|---|---|---|---|---|---|---|
|
Early Childhood Longitudinal Study-Kindergarten Class of 1998–99 (ECLS-K) Math Assessment |
Math Expressions vs. Investigations in Number, Data, and Space® |
0 Weeks |
First Grade, one-year impact;
|
44.74 |
44.51 |
No |
-- | ||
|
Early Childhood Longitudinal Study-Kindergarten Class of 1998–99 (ECLS-K) Math Assessment |
Math Expressions vs. Scott Foresman-Addison Wesley Elementary Mathematics |
0 Weeks |
First Grade, one-year impact;
|
44.74 |
44.43 |
No |
-- | ||
|
Early Childhood Longitudinal Study-Kindergarten Class of 1998–99 (ECLS-K) Math Assessment |
Math Expressions vs. Scott Foresman-Addison Wesley Elementary Mathematics |
0 Weeks |
Second Grade, one-year impact;
|
71.38 |
70.31 |
No |
-- | ||
|
Early Childhood Longitudinal Study-Kindergarten Class of 1998–99 (ECLS-K) Math Assessment |
Math Expressions vs. Saxon Math |
0 Weeks |
First Grade, one-year impact;
|
44.74 |
45.23 |
No |
-- | ||
|
Early Childhood Longitudinal Study-Kindergarten Class of 1998–99 (ECLS-K) Math Assessment |
Math Expressions vs. Investigations in Number, Data, and Space® |
0 Weeks |
Second Grade, one-year impact;
|
71.38 |
69.85 |
No |
-- | ||
|
Early Childhood Longitudinal Study-Kindergarten Class of 1998–99 (ECLS-K) Math Assessment |
Math Expressions vs. Saxon Math |
0 Weeks |
Second Grade, one-year impact;
|
71.38 |
72.53 |
No |
-- | ||
| Show Supplemental Findings | |||||||||
|
Early Childhood Longitudinal Study-Kindergarten Class of 1998–99 (ECLS-K) Math Assessment |
Math Expressions vs. Investigations in Number, Data, and Space® |
2 Years |
Second grade, two-year impact;
|
67.99 |
67.31 |
Yes |
|
||
|
Early Childhood Longitudinal Study-Kindergarten Class of 1998–99 (ECLS-K) Math Assessment |
Math Expressions vs. Saxon Math |
2 Years |
Second grade, two-year impact;
|
67.99 |
69.89 |
No |
-- | ||
|
Early Childhood Longitudinal Study-Kindergarten Class of 1998–99 (ECLS-K) Math Assessment |
Math Expressions vs. Scott Foresman-Addison Wesley Elementary Mathematics |
2 Years |
Second grade, two-year impact;
|
67.99 |
68.87 |
No |
-- | ||
Evidence Tier rating based solely on this study. This intervention may achieve a higher tier when combined with the full body of evidence.
Sample Characteristics
Characteristics of study sample as reported by study author.
-
Female: 48%
Male: 52% -
Rural, Suburban, Urban
-
- B
- A
- C
- D
- E
- F
- G
- I
- H
- J
- K
- L
- P
- M
- N
- O
- Q
- R
- S
- V
- U
- T
- W
- X
- Z
- Y
- a
- h
- i
- b
- d
- e
- f
- c
- g
- j
- k
- l
- m
- n
- o
- p
- q
- r
- s
- t
- u
- v
- x
- w
- y
Connecticut, Florida, Kentucky, Minnesota, Missouri, Mississippi, Nevada, New York, South Carolina, Texas
-
Race Asian 2% Black 33% Native American 1% Other or unknown 27% White 37% -
Ethnicity Hispanic 27% Not Hispanic or Latino 73%
Study Details
Setting
One hundred ten elementary schools in 12 districts participated in the study. The districts were located across the following 10 states: Connecticut, Florida, Kentucky, Minnesota, Mississippi, Missouri, New York, Nevada, South Carolina, and Texas. These districts are located in urban, suburban, and rural areas, and each district had at least four Title I schools (a study eligibility requirement). In total, 76% of participating schools were eligible for Title I support.
Study sample
The study sample included first- and second-grade students. Thirty-seven percent of students in participating schools were non-Hispanic White, 33% were non-Hispanic Black, 27% were Hispanic, 2% were Asian, and 1% were American Indian or Alaska Native. Just over half of the sample (52%) was male, and about half of the students were eligible for free or reduced-price lunch (51%).
Intervention Group
Math Expressions is a curriculum for students in prekindergarten through sixth grade that aims to build students’ conceptual understanding of mathematics and to develop fluency in mathematical problem solving and computation. The curriculum encourages student learning of mathematics through real-world situations, visual supports such as drawings and manipulatives, multiple approaches to solving problems, and opportunities for students to explain their mathematical thinking. Students in the intervention group used Math Expressions (published by Houghton Mifflin Harcourt) as their core math curriculum. Cohort 1 used the 2005 copyright edition, and Cohort 2 used the 2008 copyright edition. Each day began with a set of routines led by students involving the calendar, money, a number chart, counting, and time. In general, the math lesson occurred later in the day and typically began with a quick fluency activity. Afterward, the teacher provided whole-class instruction and encouraged students to discuss and demonstrate the mathematical ideas taught. Visual learning supports were used to help students link their knowledge to formal mathematical concepts. Finally, students practiced the new mathematics skill or concept in pairs, small groups, or individually by completing worksheets. Math homework was assigned daily to students. Teachers in the intervention group reported using Math Expressions as their core math curriculum and provided, on average, 5.0 to 5.5 hours of math instruction per week. Most teachers (84% in first grade and 80% in second grade) reported completing at least 80% of Math Expressions lessons.
Comparison Group
The comparison group included students using three curricula: Investigations, Saxon, and SFAW. 1. Investigations (published by Pearson) is a mathematics curriculum for students in kindergarten through fifth grade organized into units that last 2 to 5 weeks. Within each unit, the curriculum is built around two or more investigations, each providing different contexts for students to explore mathematical problems through hands-on activities, written activities, and class discussions. Classroom activities vary depending on the length and type of investigation. Teachers begin by introducing the investigation to the class through large-group, hands-on activities, and over the following days students work in smaller groups to explore the concept through in-depth problems or by playing mathematical games. At the end of the investigation, the teacher and students discuss the work completed during the investigation. Schools used the first and second editions of the textbook. Teachers provided, on average, 2.7 to 3.1 hours of math instruction per week in each grade. Additionally, most teachers (72% in first grade and 80% in second grade) reported completing at least 80% of the lessons from Investigations. 2. Saxon (published by Houghton Mifflin Harcourt) is a curriculum for students in kindergarten through fourth grade that includes scripted lesson plans for teachers. It includes explicit instruction, practice, mathematical conversations, and hands-on activities. The curriculum includes five daily activities: morning routines, fact practice, an explicit lesson, guided class practice, and homework. Morning routines are extensive and designed to reinforce the previously learned skills and lay the foundation for building new skills. In the lesson, teachers conduct a whole-class activity to introduce a new concept, using manipulatives, worksheets, or overhead transparency masters. At the end of each lesson, the teacher asks a few students to summarize for the whole class what they learned that day. Teachers provided, on average, 6.1 to 6.9 hours of math instruction per week in each grade. In addition, most teachers (87% in first and second grades) reported completing at least 80% of the lessons from Saxon. 3. SFAW (published by Pearson) is a curriculum for students in prekindergarten through sixth grade that uses a consistent daily structure. Each lesson includes six activities: a spiral review (a brief review of previously learned material), investigating the concept (hands-on exploration of a new concept), a warm-up (a brief activity to activate prior knowledge and connect it to the new lesson), teach (direct instruction of the new concept), independent practice (students practice using worksheets or manipulatives), and assessment (a concluding activity to check for understanding of the new concept). Teachers provided, on average, 5.3 to 5.5 hours of math instruction per week in each grade. Additionally, most teachers (92% in first grade and 88% in second grade) reported completing at least 80% of the lessons from SFAW.
Support for implementation
After completing random assignment, the study team connected school staff to publishers of their assigned curriculum. Research funds were used to support teacher training. The study team did not specify implementation criteria that schools needed to be meet or maintain to be included in the study after participation began. Teachers assigned to Math Expressions received 2 days of initial training in the summer before the school year began. Two follow-up trainings took place during the school year, once in the fall and again in the spring. Follow-up sessions typically consisted of classroom observations followed by short feedback sessions with teachers. Teachers assigned to Investigations received 1 day of initial training in the summer before the school year began. Trainers offered group training sessions before the start of each unit (about every 4 to 6 weeks). The follow-up sessions were typically 3 to 4 hours long and held after school. Teachers assigned to Saxon received 1 day of initial training in the summer before the school year began. One follow-up training session was tailored to meet each district’s needs and took place during the school year. Teachers assigned to SFAW received 1 day of initial training in the summer before the school year began. Follow-up training was offered about every 4 to 6 weeks throughout the school year. The follow-up trainings were typically 3 to 4 hours long and took place after school.
Additional Sources
In the case of multiple manuscripts that report on one study, the WWC selects one manuscript as the primary citation and lists other manuscripts that describe the study as additional sources.
-
Agodini, Roberto; Harris, Barbara. (2010). An Experimental Evaluation of Four Elementary School Math Curricula. Journal of Research on Educational Effectiveness, v3 n3 p199-253.
-
Agodini, Roberto; Harris, Barbara. (2016). How Teacher and Classroom Characteristics Moderate the Effects of Four Elementary Math Curricula. Elementary School Journal, v117 n2 p216-236.
-
Agodini, Roberto; Harris, Barbara; Atkins-Burnett, Sally; Heaviside, Sheila; Novak, Timothy; Murphy, Robert. (2009). Achievement Effects of Four Early Elementary School Math Curricula: Findings from First Graders in 39 Schools. NCEE 2009-4052. National Center for Education Evaluation and Regional Assistance.
-
Agodini, Roberto; Harris, Barbara; Seftor, Neil; Remillard, Janine; Thomas, Melissa. (2013). After Two Years, Three Elementary Math Curricula Outperform a Fourth. NCEE Evaluation Brief. NCEE 2013-4019. National Center for Education Evaluation and Regional Assistance.
-
Agodini, Roberto; Harris, Barbara; Remillard, Janine; Thomas, Melissa. (2013). After Two Years, Three Elementary Math Curricula Outperform a Fourth. NCEE Technical Appendix. NCEE 2013-4019. National Center for Education Evaluation and Regional Assistance.
-
Clements, Douglas H.; Agodini, Roberto; Harris, Barbara. (2013). Instructional Practices and Student Achievement: Correlations from a Study of Math Curricula. NCEE Technical Appendix. NCEE 2013-4020. National Center for Education Evaluation and Regional Assistance.
-
Remillard, J. T., Harris, B., & Agodini, R. (2014). The influence of curriculum material design on opportunities for student learning. ZDM: The International Journal on Mathematics Education, 46(5), 735-749. https://www.researchgate.net/publication/271660767_The_influence_of_curriculum_material_design_on_opportunities_for_student_learning.
-
Clements, Douglas H.; Agodini, Roberto; Harris, Barbara. (2013). Instructional Practices and Student Math Achievement: Correlations from a Study of Math Curricula. NCEE Evaluation Brief. NCEE 2013-4020. National Center for Education Evaluation and Regional Assistance.
An indicator of the effect of the intervention, the improvement index can be interpreted as the expected change in percentile rank for an average comparison group student if that student had received the intervention.
For more, please see the WWC Glossary entry for improvement index.
An outcome is the knowledge, skills, and attitudes that are attained as a result of an activity. An outcome measures is an instrument, device, or method that provides data on the outcome.
A finding that is included in the effectiveness rating. Excluded findings may include subgroups and subscales.
The sample on which the analysis was conducted.
The group to which the intervention group is compared, which may include a different intervention, business as usual, or no services.
The timing of the post-intervention outcome measure.
The number of students included in the analysis.
The mean score of students in the intervention group.
The mean score of students in the comparison group.
The WWC considers a finding to be statistically significant if the likelihood that the finding is due to chance alone, rather than a real difference, is less than five percent.
The WWC reviews studies for WWC products, Department of Education grant competitions, and IES performance measures.
The name and version of the document used to guide the review of the study.
The version of the WWC design standards used to guide the review of the study.
The result of the WWC assessment of the study. The rating is based on the strength of evidence of the effectiveness of the intervention. Studies are given a rating of Meets WWC Design Standards without Reservations, Meets WWC Design Standards with Reservations, or >Does Not Meet WWC Design Standards.
A related publication that was reviewed alongside the main study of interest.
Study findings for this report.
Based on the direction, magnitude, and statistical significance of the findings within a domain, the WWC characterizes the findings from a study as one of the following: statistically significant positive effects, substantively important positive effects, indeterminate effects, substantively important negative effects, and statistically significant negative effects. For more, please see the WWC Handbook.
The WWC may review studies for multiple purposes, including different reports and re-reviews using updated standards. Each WWC review of this study is listed in the dropdown. Details on any review may be accessed by making a selection from the drop down list.
Tier 1 Strong indicates strong evidence of effectiveness,
Tier 2 Moderate indicates moderate evidence of effectiveness, and
Tier 3 Promising indicates promising evidence of effectiveness,
as defined in the
non-regulatory guidance for ESSA
and the regulations for ED discretionary grants (EDGAR Part 77).