Impact Findings
In the first report from the evaluation (Torgesen et al. 2006), impacts on reading test scores were presented for the end of the intervention year, when the students in the evaluation were third and fifth graders. That report presented findings on student outcomes at the end of the intervention year. The second (and last) report (Torgesen et al. 2007) from the evaluation, presents estimates of impacts on reading test scores as of the end of the following year, when most of the students were fourth- and sixth-graders. Student outcomes are referred to as their "grade cohorts," that is the students' grade level when they entered the evaluation.
For purposes of this summary, the focus is on the impact of being randomly assigned to receive any intervention compared to receiving the instruction that would normally be provided. These findings are the most robust because of the larger sample sizes. The full report also estimates impacts for each of the interventions and various student subgroups. Key findings are as follows:
- The interventions improved some reading skills. For the third-grade cohort, the four interventions combined had positive impacts on phonemic decoding, word reading accuracy and fluency, and reading comprehension, although impacts were not detected for all measures of accuracy and fluency or comprehension (see Exhibit 13). For the fifth-grade cohort, the four interventions combined improved phonemic decoding on one measure, but led to a small reduction in oral reading fluency. The three word-level interventions combined had similar impacts to those for all four interventions combined, although they did not have an impact on either measure of comprehension for students in the third grade cohort. There were impacts on both measures of phonemic decoding for students in the fifth-grade cohort. For students in the third-grade cohort, Failure Free Reading (the only word level plus comprehension program) had an impact on one measure of phonemic decoding, two of the three measures of word reading accuracy and fluency, and one measure of comprehension. However, this intervention did not have any impacts for students in the fifth-grade cohort.
- The interventions did not improve PSSA scores. For the third-grade cohort, we did not detect significant impacts of the four interventions combined on reading and mathematics test scores from the Pennsylvania System of School Assessment (see Exhibit 14). For the fifth-grade cohort, the four interventions combined lowered the reading and mathematics scores.
- Younger students benefited more. The interventions generally helped students in the third-grade cohort more than students in the fifth-grade cohort (see Exhibits 13 and 14). However, the interventions did not consistently benefit any one subgroup within each grade level more than another subgroup.
- The interventions narrowed some reading gaps. The "reading gap" describes the extent to which the average 3rd or 5th grade student in the intervention group or in the control group is lagging behind the average 3rd or 5th grade student in the population. The four interventions combined generally narrowed the reading gap for students in the intervention groups compared with students in the control group for the third-grade cohort. Being in one of the interventions reduced the reading gap in Word Attack skills by about two-thirds for students in the third-grade cohort. On other word-level tests and a measure of reading comprehension, the interventions reduced the gap for students in the third-grade cohort by about one-sixth to one-third. For students in the fifth grade cohort, the interventions reduced the gap in Word Attack skills by one-half (see Exhibit 15).
The key findings presented in this report for the seven tests administered for this study one year after the interventions ended are similar to the findings from the end of the intervention year. In the earlier report (Torgesen et al. 2006) the four interventions combined and the three word-level interventions had impacts for student in the third-grade cohort on phonemic decoding, word reading accuracy and fluency, and reading comprehension. There were fewer significant impacts for students in the fifth-grade cohort than for students in the third-grade cohort.