|
|
A total of 106 studies reviewed by the WWC investigated the effects of Reading Recovery®. Four studies (Baenen, Bernhole, Dulaney, & Banks, 1997; Pinnell, DeFord, & Lyons, 1988; Pinnell, Lyons, DeFord, Bryk, & Seltzer, 1994; and Schwartz, 2005) are randomized controlled trials that meet WWC evidence standards. One study (Iverson & Tunmer, 1993) is a quasi-experimental design that meets WWC evidence standards with reservations. The remaining 101 studies do not meet either WWC evidence standards or eligibility screens.5
Baenen et al. (1997) was a randomized controlled trial that focused on first-grade students from Wake County, NC. The WWC review focuses on the outcomes of students who qualified for and were randomly assigned to either the Reading Recovery® intervention or a comparison group. From an original sample size of 168, outcomes were assessed at three time points: end of first grade (n = 147), end of second grade (n = 147), and end of third grade (n = 127). Although the WWC used only the results at the end of first grade to determine the intervention rating, information on the additional findings can be found in Appendix A4.4
Pinnell et al. (1988) was a randomized controlled trial. The study sample was composed of first-grade students in 14 schools in Columbus, OH. Students were randomly assigned to an intervention group, which received Reading Recovery® in addition to its regular classroom instruction (n = 38), or to a control group, which received an alternate compensatory program (n = 53). This comparison meets WWC evidence standards.6
Pinnell et al. (1994) was a randomized controlled trial that randomly assigned 10 low-achieving first-grade students in each of 10 Ohio schools. The WWC review focuses only on the eight schools that successfully implemented randomization for the intervention (n = 31) and comparison (n = 48) conditions.7
Schwartz (2005) was a randomized controlled trial of first-grade students from 14 states. The WWC focused on the comparison between 37 students across several schools who were randomly assigned to receive the intervention during the first half of the year, and 37 students who were randomly assigned to receive the intervention during the second half of the year.8 The groups were compared at midyear, before the comparison group had begun receiving Reading Recovery®.
Iverson and Tunmer (1993) was a quasi-experimental design study that included first-grade students from 30 school districts in Rhode Island. The study compared outcomes for students participating in Reading Recovery® (n = 32) with students in a comparison group who did not receive Reading Recovery® (n = 32), who were matched on the basis of pretest scores.9 The comparison group received standard small group, out-of-class support services.
The WWC categorizes the extent of evidence in each domain as small or medium to large (see the What Works Clearinghouse Extent of Evidence Categorization Scheme). The extent of evidence takes into account the number of studies and the total sample size across the studies that meet WWC evidence standards with or without reservations.10
The WWC considers the extent of evidence for Reading Recovery® to be medium to large for alphabetics, small for fluency and comprehension, and medium to large for general reading achievement.
5 Because Reading Recovery® is designed to improve the reading skills of low-achieving first-grade readers, the appropriate comparison groups for determining the intervention’s effectiveness are similar low achieving first-grade readers who did not receive Reading Recovery®. Many of the studies screened did not meet evidence standards because they used inappropriate comparison groups, such as higher-achieving first-grade readers, to draw conclusions about the effectiveness of the program.
6 A third group of students qualified for and received Reading Recovery® outside of regular classroom instruction, but these students were also taught by a Reading Recovery®–trained teacher when they were in their regular classroom (n = 96). Although this comparison met evidence standards with reservations, it was not considered in the intervention rating because it went beyond the standard delivery of the program. However, results are reported in Appendices A4.1–A4.3.
7 Although the original study included analyses of additional interventions implemented at additional schools, only the schools that randomly assigned students to Reading Recovery® or the comparison group were relevant to this review. For more details about the original study, see Appendix A1.3.
8 Assessments were also made at the end of the year, but they were not appropriate for the WWC’s analysis because by then both groups of low-achieving students had received the intervention. Additional comparison groups of low-average and high-average readers were not used by the WWC because these students were not eligible for Reading Recovery®.
9 The study also included a third group of students (n = 32) who used a modified version of Reading Recovery®, which provided explicit instruction in letter-phoneme patterns instead of the letter identification segment. This group was also compared with the comparison group. Although this comparison meets evidence standards with reservations, it was not considered in the intervention rating because it went beyond the standard delivery of the program. However, results are reported in Appendices A4.1 and A4.3.
10 The Extent of Evidence Categorization was developed to tell readers how much evidence was used to determine the intervention rating, focusing on the number and size of studies. Additional factors associated with a related concept—external validity, such as the students' demographics and the types of settings in which studies took place—are not taken into account for the categorization. Information about how the extent of evidence rating was determined for Reading Recovery® is in Appendix A6.
|Institute of Education Sciences
