Agodini, Roberto; Harris, Barbara; Thomas, Melissa; Murphy, Robert; Gallagher, Lawrence (2010). National Center for Education Evaluation and Regional Assistance. Retrieved from: https://eric.ed.gov/?id=ED512551
Reviewed: April 2017
This review may not reflect the full body of research evidence for this intervention.
Evidence Tier rating based solely on this study. This intervention may achieve a higher tier when combined with the full body of evidence.
Please see the WWC summary of evidence for Saxon Math.
Reviewed: June 2016
This review may not reflect the full body of research evidence for this intervention.
Evidence Tier rating based solely on this study. This intervention may achieve a higher tier when combined with the full body of evidence.
Please see the WWC summary of evidence for enVisionMATH.
Reviewed: May 2016
This review may not reflect the full body of research evidence for this intervention.
Evidence Tier rating based solely on this study. This intervention may achieve a higher tier when combined with the full body of evidence.
Please see the WWC summary of evidence for Saxon Algebra I.
Reviewed: May 2013
This review may not reflect the full body of research evidence for this intervention.
Evidence Tier rating based solely on this study. This intervention may achieve a higher tier when combined with the full body of evidence.
Please see the WWC summary of evidence for Investigations in Number, Data, and Space®.
|
Outcome measure |
Comparison | Period | Sample |
Intervention mean |
Comparison mean |
Significant? |
Improvement index |
Evidence tier |
|
|---|---|---|---|---|---|---|---|---|---|
|
Early Childhood Longitudinal Study- Kindergarten (ECLS-K) Math Assessment |
Investigations in Number, Data, and Space® vs. Scott Foresman-Addison Wesley Mathematics |
Posttest |
Grade 2;
|
71.66 |
70.31 |
No |
-- | ||
|
Early Childhood Longitudinal Study- Kindergarten (ECLS-K) Math Assessment |
Investigations in Number, Data, and Space® vs. Scott Foresman-Addison Wesley Mathematics |
Posttest |
Grade 1;
|
44.40 |
44.43 |
No |
-- | ||
|
Early Childhood Longitudinal Study- Kindergarten (ECLS-K) Math Assessment |
Investigations in Number, Data, and Space® vs. Math Expressions |
Posttest |
Grade 2;
|
70.84 |
71.38 |
No |
-- | ||
|
Early Childhood Longitudinal Study- Kindergarten (ECLS-K) Math Assessment |
Investigations in Number, Data, and Space® vs. Saxon math |
Posttest |
Grade 1;
|
44.69 |
45.23 |
No |
-- | ||
|
Early Childhood Longitudinal Study- Kindergarten (ECLS-K) Math Assessment |
Investigations in Number, Data, and Space® vs. Math Expressions |
Posttest |
Grade 1;
|
43.82 |
44.74 |
No |
-- | ||
|
Early Childhood Longitudinal Study- Kindergarten (ECLS-K) Math Assessment |
Investigations in Number, Data, and Space® vs. Saxon Math |
Posttest |
Grade 2;
|
71.13 |
72.53 |
No |
-- | ||
| Show Supplemental Findings | |||||||||
|
Early Childhood Longitudinal Study- Kindergarten (ECLS-K) Math Assessment |
Investigations in Number, Data, and Space® vs. Scott Foresman-Addison Wesley Mathematics |
Posttest |
Grade 2: Schools with up to 40% eligibility for FRPL;
|
N/A |
N/A |
No |
-- | ||
|
Early Childhood Longitudinal Study- Kindergarten (ECLS-K) Math Assessment |
Investigations in Number, Data, and Space® vs. Scott Foresman-Addison Wesley Mathematics |
Posttest |
Grade 1: Schools with greater than 40% eligibility for FRPL;
|
N/A |
N/A |
No |
-- | ||
|
Early Childhood Longitudinal Study- Kindergarten (ECLS-K) Math Assessment |
Investigations in Number, Data, and Space® vs. Math Expressions |
Posttest |
Grade 2: Schools with up to 40% eligibility for FRPL;
|
N/A |
N/A |
No |
-- | ||
|
Early Childhood Longitudinal Study- Kindergarten (ECLS-K) Math Assessment |
Investigations in Number, Data, and Space® vs. Scott Foresman-Addison Wesley Mathematics |
Posttest |
Grade 1: Schools with up to 40% eligibility for FRPL;
|
N/A |
N/A |
No |
-- | ||
|
Early Childhood Longitudinal Study- Kindergarten (ECLS-K) Math Assessment |
Investigations in Number, Data, and Space® vs. Scott Foresman-Addison Wesley Mathematics |
Posttest |
Grade 2: Schools with greater than 40% eligibility for FRPL;
|
N/A |
N/A |
No |
-- | ||
|
Early Childhood Longitudinal Study- Kindergarten (ECLS-K) Math Assessment |
Investigations in Number, Data, and Space® vs. Math Expressions |
Posttest |
Grade 1: Schools with greater than 40% eligibility for FRPL;
|
N/A |
N/A |
No |
-- | ||
|
Early Childhood Longitudinal Study- Kindergarten (ECLS-K) Math Assessment |
Investigations in Number, Data, and Space® vs. Saxon Math |
Posttest |
Grade 1: Schools with greater than 40% eligibility for FRPL;
|
N/A |
N/A |
No |
-- | ||
|
Early Childhood Longitudinal Study- Kindergarten (ECLS-K) Math Assessment |
Investigations in Number, Data, and Space® vs. Saxon Math |
Posttest |
Grade 1: Schools with up to 40% eligibility for FRPL;
|
N/A |
N/A |
No |
-- | ||
|
Early Childhood Longitudinal Study- Kindergarten (ECLS-K) Math Assessment |
Investigations in Number, Data, and Space® vs. Math Expressions |
Posttest |
Grade 1: Schools with up to 40% eligibility for FRPL;
|
N/A |
N/A |
No |
-- | ||
|
Early Childhood Longitudinal Study- Kindergarten (ECLS-K) Math Assessment |
Investigations in Number, Data, and Space® vs. Saxon Math |
Posttest |
Grade 2: Schools with greater than 40% eligibility for FRPL;
|
N/A |
N/A |
No |
-- | ||
|
Early Childhood Longitudinal Study- Kindergarten (ECLS-K) Math Assessment |
Investigations in Number, Data, and Space® vs. Saxon Math |
Posttest |
Grade 2: Schools with up to 40% eligibility for FRPL;
|
N/A |
N/A |
Yes |
|
||
|
Early Childhood Longitudinal Study- Kindergarten (ECLS-K) Math Assessment |
Investigations in Number, Data, and Space® vs. Math Expressions |
Posttest |
Grade 2: Schools with greater than 40% eligibility for FRPL;
|
N/A |
N/A |
No |
-- | ||
Characteristics of study sample as reported by study author.
| Asian | 2% | |
| Black | 33% | |
| Native American | 1% | |
| White | 37% |
| Hispanic | 27% | |
| Not Hispanic or Latino | 73% |
Reviewed: May 2013
This review may not reflect the full body of research evidence for this intervention.
Evidence Tier rating based solely on this study. This intervention may achieve a higher tier when combined with the full body of evidence.
Please see the WWC summary of evidence for Scott Foresman-Addison Wesley Elementary Mathematics.
|
Outcome measure |
Comparison | Period | Sample |
Intervention mean |
Comparison mean |
Significant? |
Improvement index |
Evidence tier |
|---|---|---|---|---|---|---|---|---|
|
Early Childhood Longitudinal Study- Kindergarten (ECLS-K) Math Assessment |
Scott Foresman-Addison Wesley Elementary Mathematics vs. Investigations in Number, Data, and Space |
Spring Semester |
Grade 1;
|
44.54 |
44.51 |
No |
-- | |
|
Early Childhood Longitudinal Study- Kindergarten (ECLS-K) Math Assessment |
Scott Foresman-Addison Wesley Elementary Mathematics vs. Saxon math |
Spring Semester |
Grade 1;
|
44.72 |
45.23 |
No |
-- | |
|
Early Childhood Longitudinal Study- Kindergarten (ECLS-K) Math Assessment |
Scott Foresman-Addison Wesley Elementary Mathematics vs. Math Expressions |
Spring Semester |
Grade 1;
|
43.85 |
44.74 |
Yes |
|
|
|
Early Childhood Longitudinal Study- Kindergarten (ECLS-K) Math Assessment |
Scott Foresman-Addison Wesley Elementary Mathematics vs. Investigations in Number, Data, and Space |
Spring Semester |
Grade 2;
|
68.50 |
69.85 |
No |
-- | |
|
Early Childhood Longitudinal Study- Kindergarten (ECLS-K) Math Assessment |
Scott Foresman-Addison Wesley Elementary Mathematics vs. Math Expressions |
Spring Semester |
Grade 2;
|
69.49 |
71.38 |
Yes |
|
|
|
Early Childhood Longitudinal Study- Kindergarten (ECLS-K) Math Assessment |
Scott Foresman-Addison Wesley Elementary Mathematics vs. Saxon Math |
Spring Semester |
Grade 2;
|
69.78 |
72.53 |
Yes |
|
Characteristics of study sample as reported by study author.
| Asian | 2% | |
| Black | 32% | |
| Native American | 1% | |
| White | 39% |
| Hispanic | 26% | |
| Not Hispanic or Latino | 74% |
The study took place in elementary schools in 12 districts across 10 states, including Connecticut, Florida, Kentucky, Minnesota, Mississippi, Missouri, Nevada, New York, South Carolina, and Texas. Of the 12 districts, three were in urban areas, five were in suburban areas, and four were in rural areas.
Following district and school recruitment and collection of consent from all teachers in the participating grades, 111 participating schools were randomly assigned to one of four curricula: (a) Investigations in Number, Data, and Space®, (b) Math Expressions, (c) Saxon Math, and (d) Scott Foresman–Addison Wesley Mathematics. Blocked random assignment of the schools was conducted separately within each district. In each district, participating schools were grouped together into blocks of four to seven schools based on characteristics such as Title I eligibility, free or reduced-price lunch eligibility status, grade enrollment size, math proficiency, and proportion of White and Hispanic students. Two districts had an additional blocking variable (magnet school status in one district and year-round school schedule in another district). One district required that all schools that fed into the same middle school receive the same condition. Schools in each block were randomly assigned among the four curricula. On average, 11 students were randomly sampled from each participating classroom for assessment. One school with three teachers and 32 students assigned to Math Expressions withdrew from the study and did not permit follow-up data collection. The analysis sample included a total of 110 schools, 461 first-grade classrooms, 4,716 first graders, 328 second-grade classrooms, and 3,344 second graders. In the first grade sample, on average, 27 schools, 116 classrooms, and 1,180 students were assigned to each condition. In the second grade sample, on average, 18 schools, 82 classrooms, and 835 students were assigned to each condition. Seventy-six percent of the schools in the study were eligible for Title I funding. Approximately half of the students in the sample were eligible for free or reduced-price lunch. Among students in the sample, 39% were White, 32% were non-Hispanic Black, 26% were Hispanic, 2% were Asian, and 1% were American Indian or Alaskan Native.
Students in the intervention group used Scott Foresman–Addison Wesley Mathematics as their core math curriculum. Study authors reported about nine out of 10 teachers self-reported completing at least 80% of the curriculum.
The study included three comparison groups: (a) Investigations in Number, Data, and Space®, (b) Math Expressions, and (c) Saxon Math. Each curriculum was implemented by comparison teachers for 1 school year. Investigations in Number, Data, and Space® is published by Pearson Scott Foresman. It uses a student-centered approach that encourages reasoning and understanding and draws on constructivist learning theory. The lessons build on students’ existing knowledge and focus on understanding math concepts rather than simply learning computational methods. The curriculum is organized in nine thematic units, each lasting 2–5.5 weeks. Study authors reported that about four out of five teachers self-reported completing at least 80% of the curriculum. Math Expressions is published by Houghton Mifflin Harcourt and uses a blend of student-centered and teacher-directed instructional approaches. Students using the curriculum question and discuss mathematics and are explicitly taught problem solving strategies. There is an emphasis on using multiple specified objects, drawings, and language to represent concepts, and on learning through the use of real-world situations. Students are expected to explain and justify their solutions. Study authors reported that about nine out of 10 teachers self-reported completing at least 80% of the curriculum. Saxon Math is published by Houghton Mifflin Harcourt and uses a teacher-directed approach that offers a script for teachers to follow in each lesson. It blends teacher-directed instruction of new material with daily practice of previously learned concepts and procedures. The teacher introduces concepts or efficient strategies for solving problems. Students receive instruction from the teacher, participate in guided practice, and then undertake individual practice. Frequent monitoring of student achievement is built into the program. Daily routines are extensive and emphasize practice of number concepts and use of methods (such as the use of number lines, counting on fingers, and diagrams) to represent mathematical concepts. Study authors reported that about six out of seven teachers self-reported completing at least 80% of the curriculum.
Mathematics achievement was measured using the mathematics assessment developed for the ECLS-K class of 1998–99. The assessment is individually administered, nationally normed, and adaptive. The assessment meets accepted standards of validity and reliability. Scale scores from an item response theory (IRT) model were used in the analysis. The test was administered in the fall of the implementation year (within 4 weeks of the first day of classes) to assess students’ baseline math achievement. The test was also administered in the spring—that is, from 1–6 weeks before the end of the school year of program implementation. For a more detailed description of the outcome measure, see Appendix B.
Teachers in all four groups were provided training by the curriculum publisher. Teachers assigned to Scott Foresman–Addison Wesley Elementary Mathematics received 1 day of initial training in the summer before the school year began. Follow-up training was offered about every 4–6 weeks throughout the school year. Follow-up sessions were typically 3–4 hours long and held after school. Teachers assigned to Investigations in Number, Data, and Space® (comparison group 1) were provided 1 day of initial training in the summer before the school year began. Follow-up sessions were typically 3–4 hours long and held after school. Teachers assigned to Math Expressions (comparison group 2) were provided 2 days of initial training in the summer before the school year began. Two follow-up trainings were offered during the school year. Follow-up sessions typically consisted of classroom observations followed by short feedback sessions with teachers. Teachers assigned to Saxon Math (comparison group 3) were provided 1 day of initial training in the summer before the school year began. One follow-up training session, tailored to meet each district’s needs, was offered during the school year.
In the case of multiple manuscripts that report on one study, the WWC selects one manuscript as the primary citation and lists other manuscripts that describe the study as additional sources.
Agodini, Roberto; Harris, Barbara. (2010). An Experimental Evaluation of Four Elementary School Math Curricula. Journal of Research on Educational Effectiveness, v3 n3 p199-253.
Agodini, Roberto; Harris, Barbara; Atkins-Burnett, Sally; Heaviside, Sheila; Novak, Timothy; Murphy, Robert. (2009). Achievement Effects of Four Early Elementary School Math Curricula: Findings from First Graders in 39 Schools. NCEE 2009-4052. National Center for Education Evaluation and Regional Assistance.
Agodini, R., Harris, B., Atkins-Burnett, S., Heaviside, S., Novak, T., Murphy, R., & Institute of Education Sciences, National Center for Education Evaluation and Regional Assistance. (2009). Achievement effects of four early elementary school math curricula: Findings from first graders in 39 schools. Executive summary (NCEE 2009-4053). Washington, DC: National Center for Education Evaluation and Regional Assistance, Institute of Education Sciences, U.S. Department of Education.
Reviewed: April 2011
This review may not reflect the full body of research evidence for this intervention.
Evidence Tier rating based solely on this study. This intervention may achieve a higher tier when combined with the full body of evidence.
|
Outcome measure |
Comparison | Period | Sample |
Intervention mean |
Comparison mean |
Significant? |
Improvement index |
Evidence tier |
|---|---|---|---|---|---|---|---|---|
|
Early Childhood Longitudinal Study- Kindergarten (ECLS-K) Math Assessment |
Math Expressions vs. Scott Foresman-Addison Wesley Mathematics |
Posttest |
Grade 2;
|
15.14 |
14.32 |
Yes |
|
|
|
Early Childhood Longitudinal Study- Kindergarten (ECLS-K) Math Assessment |
Math Expressions vs. Scott Foresman-Addison Wesley Mathematics |
Posttest |
Grade 1;
|
14.16 |
13.25 |
Yes |
|
|
|
Early Childhood Longitudinal Study- Kindergarten (ECLS-K) Math Assessment |
Math Expressions vs. Saxon |
Posttest |
Grade 1;
|
14.16 |
13.56 |
No |
-- | |
|
Early Childhood Longitudinal Study- Kindergarten (ECLS-K) Math Assessment |
Math Expressions vs. Saxon |
Posttest |
Grade 2;
|
15.14 |
16.59 |
No |
-- |
Characteristics of study sample as reported by study author.
| Asian | 2% | |
| Black | 32% | |
| Native American | 1% | |
| White | 39% |
| Hispanic | 26% | |
| Not Hispanic or Latino | 74% |
Reviewed: April 2011
This review may not reflect the full body of research evidence for this intervention.
Evidence Tier rating based solely on this study. This intervention may achieve a higher tier when combined with the full body of evidence.
|
Outcome measure |
Comparison | Period | Sample |
Intervention mean |
Comparison mean |
Significant? |
Improvement index |
Evidence tier |
|---|---|---|---|---|---|---|---|---|
|
Early Childhood Longitudinal Study- Kindergarten (ECLS-K) Math Assessment |
Saxon Math vs. Scott Foresman-Addison Wesley Mathematics |
Posttest |
Grade 2;
|
16.59 |
14.32 |
Yes |
|
|
|
Early Childhood Longitudinal Study- Kindergarten (ECLS-K) Math Assessment |
Saxon Math vs. Scott Foresman-Addison Wesley Mathematics |
Posttest |
Grade 1;
|
13.56 |
13.25 |
No |
-- |
Characteristics of study sample as reported by study author.
| Asian | 2% | |
| Black | 32% | |
| Native American | 1% | |
| White | 39% |
| Hispanic | 26% | |
| Not Hispanic or Latino | 74% |
Reviewed: April 2011
This review may not reflect the full body of research evidence for this intervention.
Evidence Tier rating based solely on this study. This intervention may achieve a higher tier when combined with the full body of evidence.
|
Outcome measure |
Comparison | Period | Sample |
Intervention mean |
Comparison mean |
Significant? |
Improvement index |
Evidence tier |
|---|---|---|---|---|---|---|---|---|
|
Early Childhood Longitudinal Study- Kindergarten (ECLS-K) Math Assessment |
Investigations in Number, Data, and Space® vs. Scott Foresman-Addison Wesley Mathematics |
Posttest |
Grade 2;
|
14.81 |
14.32 |
No |
-- | |
|
Early Childhood Longitudinal Study- Kindergarten (ECLS-K) Math Assessment |
Investigations in Number, Data, and Space® vs. Scott Foresman-Addison Wesley Mathematics |
Posttest |
Grade 1;
|
13.36 |
13.25 |
No |
-- | |
|
Early Childhood Longitudinal Study- Kindergarten (ECLS-K) Math Assessment |
Investigations in Number, Data, and Space® vs. Math Expressions |
Posttest |
Grade 2;
|
14.81 |
15.14 |
No |
-- | |
|
Early Childhood Longitudinal Study- Kindergarten (ECLS-K) Math Assessment |
Investigations in Number, Data, and Space® vs. Saxon |
Posttest |
Grade 1;
|
13.36 |
13.56 |
No |
-- | |
|
Early Childhood Longitudinal Study- Kindergarten (ECLS-K) Math Assessment |
Investigations in Number, Data, and Space® vs. Math Expressions |
Posttest |
Grade 1;
|
13.36 |
14.16 |
Yes |
|
|
|
Early Childhood Longitudinal Study- Kindergarten (ECLS-K) Math Assessment |
Investigations in Number, Data, and Space® vs. Saxon |
Posttest |
Grade 2;
|
14.81 |
16.59 |
No |
-- |
Characteristics of study sample as reported by study author.
| Asian | 2% | |
| Black | 32% | |
| Native American | 1% | |
| White | 39% |
| Hispanic | 26% | |
| Not Hispanic or Latino | 74% |
An indicator of the effect of the intervention, the improvement index can be interpreted as the expected change in percentile rank for an average comparison group student if that student had received the intervention.
For more, please see the WWC Glossary entry for improvement index.
An outcome is the knowledge, skills, and attitudes that are attained as a result of an activity. An outcome measures is an instrument, device, or method that provides data on the outcome.
A finding that is included in the effectiveness rating. Excluded findings may include subgroups and subscales.
The sample on which the analysis was conducted.
The group to which the intervention group is compared, which may include a different intervention, business as usual, or no services.
The timing of the post-intervention outcome measure.
The number of students included in the analysis.
The mean score of students in the intervention group.
The mean score of students in the comparison group.
The WWC considers a finding to be statistically significant if the likelihood that the finding is due to chance alone, rather than a real difference, is less than five percent.
The WWC reviews studies for WWC products, Department of Education grant competitions, and IES performance measures.
The name and version of the document used to guide the review of the study.
The version of the WWC design standards used to guide the review of the study.
The result of the WWC assessment of the study. The rating is based on the strength of evidence of the effectiveness of the intervention. Studies are given a rating of Meets WWC Design Standards without Reservations, Meets WWC Design Standards with Reservations, or >Does Not Meet WWC Design Standards.
A related publication that was reviewed alongside the main study of interest.
Study findings for this report.
Based on the direction, magnitude, and statistical significance of the findings within a domain, the WWC characterizes the findings from a study as one of the following: statistically significant positive effects, substantively important positive effects, indeterminate effects, substantively important negative effects, and statistically significant negative effects. For more, please see the WWC Handbook.
The WWC may review studies for multiple purposes, including different reports and re-reviews using updated standards. Each WWC review of this study is listed in the dropdown. Details on any review may be accessed by making a selection from the drop down list.
Tier 1 Strong indicates strong evidence of effectiveness,
Tier 2 Moderate indicates moderate evidence of effectiveness, and
Tier 3 Promising indicates promising evidence of effectiveness,
as defined in the
non-regulatory guidance for ESSA
and the regulations for ED discretionary grants (EDGAR Part 77).
Resources
Newsflash
Contact Us
Facebook
Twitter
YouTube