2005 Scott Foresman–Addison Wesley Elementary Math randomized control trial: Final report.
Resendez, M., & Azin, M. (2006). Jackson, WY: PRES Associates, Inc.
-
examining862Students, grades3-5
Scott Foresman-Addison Wesley Elementary Mathematics Intervention Report - Elementary School Mathematics
Review Details
Reviewed: May 2013
- Randomized Controlled Trial
- Meets WWC standards without reservations
This review may not reflect the full body of research evidence for this intervention.
Evidence Tier rating based solely on this study. This intervention may achieve a higher tier when combined with the full body of evidence.
Please see the WWC summary of evidence for Scott Foresman-Addison Wesley Elementary Mathematics.
Findings
|
Outcome measure |
Comparison | Period | Sample |
Intervention mean |
Comparison mean |
Significant? |
Improvement index |
Evidence tier |
|---|---|---|---|---|---|---|---|---|
|
TerraNova Comprehensive Tests of Basic Skills (CTBS): Basic Mulitple Assessment Plus (Math Computation) |
Scott Foresman-Addison Wesley Elementary Mathematics vs. Unknown |
Posttest |
Grades 3 and 5;
|
633.28 |
624.83 |
Yes |
|
|
|
TerraNova Comprehensive Tests of Basic Skills (CTBS): Basic Mulitple Assessment (Math Total) |
Scott Foresman-Addison Wesley Elementary Mathematics vs. Unknown |
Posttest |
Grades 3 and 5;
|
654.71 |
656.00 |
No |
-- |
Evidence Tier rating based solely on this study. This intervention may achieve a higher tier when combined with the full body of evidence.
Sample Characteristics
Characteristics of study sample as reported by study author.
-
Female: 49%
Male: 51% -
Suburban, Urban
-
- B
- A
- C
- D
- E
- F
- G
- I
- H
- J
- K
- L
- P
- M
- N
- O
- Q
- R
- S
- V
- U
- T
- W
- X
- Z
- Y
- a
- h
- i
- b
- d
- e
- f
- c
- g
- j
- k
- l
- m
- n
- o
- p
- q
- r
- s
- t
- u
- v
- x
- w
- y
New Jersey, Ohio
-
Race Asian 27% Black 9% White 58% -
Ethnicity Hispanic 6% Not Hispanic or Latino 94%
Study Details
Setting
Four schools located in urban and suburban settings participated in the study. Two schools were located in Ohio and two schools were located in New Jersey.
Study sample
Third- and fifth-grade teachers were randomly assigned within schools to either the intervention or comparison condition. The baseline sample included 39 teachers (20 intervention and 19 comparison) and 915 students (468 intervention and 447 comparison). Twenty-three teachers taught third grade (13 intervention and 10 comparison), and 16 taught fifth grade (seven intervention and nine comparison). No teachers left the study, and student attrition was low. Between 837 and 863 students were tested at the end of the school year on the TerraNova Comprehensive Tests of Basics Skills (CTBS) Basic Multiple Assessment (Math Total) and TerraNova CTBS Basic Multiple Assessment Plus (Math Computation). On average, participating schools had a lower percentage of Hispanic and African-American students, special education students, and students eligible for free or reduced-price meals than the national average. These schools had higher average percentages of Asian students and students with higher ability levels than the national average.
Intervention Group
Students used the 2005 Scott Foresman–Addison Wesley Elementary Mathematics curriculum during the 2005–06 school year. The program was implemented according to the curricula guidelines. Implementation was monitored throughout the school year using online teacher logs and classroom observation. The study authors reported that teachers covered 79% of the curriculum on average.
Comparison Group
Comparison students used three different math curricula. Students in two schools used a chapter-based, comprehensive basal program. Students in a third school used a different basal math program that placed greater emphasis on repetitive, sequential review and regular assessments. Students in a fourth school used a school-created math program that was based on a number of different math materials from various resources. The comparison curricula generally covered the same content as Scott Foresman–Addison Wesley Elementary Mathematics. The study authors reported that teachers covered 80% of the curricula on average.
Outcome descriptions
The authors administered the TerraNova Basic Multiple Assessment Plus test (Level 13 in third grade and Level 15 in fifth grade). The math test provides two overall scores: the TerraNova Comprehensive Tests of Basics Skills (CTBS) Basic Multiple Assessment (Math Total) and the TerraNova CTBS Basic Multiple Assessment Plus (Math Computation) Total. The Math Total score is based on multiple choice and constructed response items that are predominantly word problems that measure basic, applied, and higher-order thinking skills. The Math Computation Total is based on the Plus test booklet, which contains only multiple-choice computational problems. Scale scores were used in the analysis. For a more detailed description of these outcome measures, see Appendix B.
Support for implementation
Teachers received 3 hours of initial training prior to implementing Scott Foresman–Addison Wesley Elementary Mathematics in their classes. At the initial training session, the trainer described the key components of the curriculum, reviewed the teacher’s edition textbook and available ancillary resources, offered examples of when to use certain materials, provided an overview of the math technology available, and modeled a math lesson. The training focused on the components most vital to the program and those that were required for full implementation. Two follow-up sessions were offered during the school year. The first was offered 4–8 weeks into the school year and lasted 2 hours. The session was informal and allowed teachers to discuss and ask questions about implementation issues. A second follow-up session, addressing pacing issues and further covering the technology available with the program, was provided to one school in March. The other three schools were offered the second follow-up session but chose not to receive it.
Additional Sources
In the case of multiple manuscripts that report on one study, the WWC selects one manuscript as the primary citation and lists other manuscripts that describe the study as additional sources.
-
Resendez, M., & Sridiharan, S. (2006). A study on the effectiveness of the 2005 Scott Foresman Addison–Wesley Elementary Math program: Technical report. Jackson, WY: PRES Associates, Inc.
An indicator of the effect of the intervention, the improvement index can be interpreted as the expected change in percentile rank for an average comparison group student if that student had received the intervention.
For more, please see the WWC Glossary entry for improvement index.
An outcome is the knowledge, skills, and attitudes that are attained as a result of an activity. An outcome measures is an instrument, device, or method that provides data on the outcome.
A finding that is included in the effectiveness rating. Excluded findings may include subgroups and subscales.
The sample on which the analysis was conducted.
The group to which the intervention group is compared, which may include a different intervention, business as usual, or no services.
The timing of the post-intervention outcome measure.
The number of students included in the analysis.
The mean score of students in the intervention group.
The mean score of students in the comparison group.
The WWC considers a finding to be statistically significant if the likelihood that the finding is due to chance alone, rather than a real difference, is less than five percent.
The WWC reviews studies for WWC products, Department of Education grant competitions, and IES performance measures.
The name and version of the document used to guide the review of the study.
The version of the WWC design standards used to guide the review of the study.
The result of the WWC assessment of the study. The rating is based on the strength of evidence of the effectiveness of the intervention. Studies are given a rating of Meets WWC Design Standards without Reservations, Meets WWC Design Standards with Reservations, or >Does Not Meet WWC Design Standards.
A related publication that was reviewed alongside the main study of interest.
Study findings for this report.
Based on the direction, magnitude, and statistical significance of the findings within a domain, the WWC characterizes the findings from a study as one of the following: statistically significant positive effects, substantively important positive effects, indeterminate effects, substantively important negative effects, and statistically significant negative effects. For more, please see the WWC Handbook.
The WWC may review studies for multiple purposes, including different reports and re-reviews using updated standards. Each WWC review of this study is listed in the dropdown. Details on any review may be accessed by making a selection from the drop down list.
Tier 1 Strong indicates strong evidence of effectiveness,
Tier 2 Moderate indicates moderate evidence of effectiveness, and
Tier 3 Promising indicates promising evidence of effectiveness,
as defined in the
non-regulatory guidance for ESSA
and the regulations for ED discretionary grants (EDGAR Part 77).