Evaluation of the i3 scale-up of Reading Recovery year one report, 2011–12.
May, H., Gray, A., Gillespie, J. N., Sirinides, P., Sam, C., Goldsworthy, H., Armijo, M., & Tognatta, N. (2013). Philadelphia, PA: Consortium for Policy Research in Education. Retrieved from: https://eric.ed.gov/?id=ED547669
-
examining866Students, grade1
Practice Guide
Review Details
Reviewed: February 2023
- Practice Guide (findings for Reading Recovery (RR))
- Randomized Controlled Trial
- Meets WWC standards without reservations because it is a randomized controlled trial with low attrition.
This review may not reflect the full body of research evidence for this intervention.
Evidence Tier rating based solely on this study. This intervention may achieve a higher tier when combined with the full body of evidence.
Findings
|
Outcome measure |
Comparison | Period | Sample |
Intervention mean |
Comparison mean |
Significant? |
Improvement index |
Evidence tier |
|---|---|---|---|---|---|---|---|---|
|
Iowa Test of Basic Skills- Reading Comprehension Subtest |
Reading Recovery (RR) vs. Business as usual |
0 Days |
Full sample;
|
140.01 |
135.50 |
Yes |
|
|
Evidence Tier rating based solely on this study. This intervention may achieve a higher tier when combined with the full body of evidence.
Sample Characteristics
Study sample characteristics were not reported.Study Details
Setting
The Ohio State University received a grant from the US department of education in 2010 to scale up the use of Reading Recovery at schools across the United States. The study took place at a sample of schools receiving i3 scale-up grants. The schools randomly assigned 1st grade students who were struggling readers as of the beginning of the school year to either receive Reading Recovery or normal classroom instruction.
Study sample
"Demographic data for gender, ELL status, and race were presented. These data were only available for 862, 860, and 856 students from the analysis sample, respectively. There were no significant differences in these characteristics between students in the treatment and control groups. Free/reduced price lunch data were not available, though these data are being collected for the 2012-13 year of the evaluation. In the treatment group 61% of the students were male, 17% were ELL, 18% were Black, 22% were Hispanic, 57% were White, and 3% were categorized as other race. In the control group 61% of the students were male, 18% were ELL, 19% were Black, 20% were Hispanic, 56% were White, and 5% were categorized as other race."
Intervention Group
"The intervention, Reading Recovery (RR), is intended to last between 12 and 20 weeks. The intervention involves a daily 30 minute pull-out session of one-on-one instruction with a trained reading recovery teacher (who is not the child's normal 1st grade teacher). Each RR teacher was supposed to work with 4 students each day. Teachers successfully worked with all four students 73 percent of the time, with most failures to work with all four students resulting from student absences. Each RR session began ""... with re-reading familiar books and a running record."" This was followed by ""... word or letter work on the wallboard; story composition; assembling a cut-up sentence; and finally previewing and reading a new book."" The teachers surveyed in the study reported high fidelity in the implementation of RR lessons. As implemented: Students worked one-on-one with a Reading Recovery teacher for 30 minutes each day over a period that varied from 12 to 20 weeks. The sessions, which were tailored to each individual student's needs as determined by frequent progress monitoring, included re-reading familiar books, word or letter work on a wallboard, story composition, assembling sentences from a cut-up story, and previewing and reading a new book. The location of the sessions (in the regular classroom or pull-out) was not reported."
Comparison Group
Students in the control group continued to receive normal classroom instruction and were not pulled out for the one-on-one sessions with RR teachers during the intervention period. After the mid-year administration of the post-test, students in the comparison group were eligible to receive instruction in RR during the remainder of the school year.
Support for implementation
Reading Recovery (RR) teachers participated in training sessions at designated facilities or at the schools where the teachers worked. The teachers were expected to learn how to design individualized daily lessons and deliver these lessons, document the lessons, and to collect and make use of data on student progress. The teacher learning was supported in three main ways: (1) Teachers completed a one week course in the summer that addressed the interpretation and scoring of the Observation Survey of Early Literacy Achievement (the pre-test given to students in the evaluation to assess their reading level). (2) Teachers completed an 8-10 credit year long academic course taught by a RR teacher leader. During this course RR teachers provided one-on-one lessons to four RR students and attend weekly 3-hour training sessions run by their teacher leader. (3) RR teachers received visits from their teacher leader during which the leader observed their RR lessons and provided feedback.
Evaluation of the i3 scale-up of Reading Recovery year one report, 2011–12.
Review Details
Reviewed: October 2014
- Single Study Review (114 KB) (findings for Reading Recovery®)
- Randomized Controlled Trial
- Meets WWC standards without reservations
This review may not reflect the full body of research evidence for this intervention.
Evidence Tier rating based solely on this study. This intervention may achieve a higher tier when combined with the full body of evidence.
Findings
|
Outcome measure |
Comparison | Period | Sample |
Intervention mean |
Comparison mean |
Significant? |
Improvement index |
Evidence tier |
|---|---|---|---|---|---|---|---|---|
|
Iowa Test of Basic Skills (ITBS): Reading |
Reading Recovery® vs. None |
post intervention |
First-grade students;
|
139.24 |
135.00 |
Yes |
|
|
Evidence Tier rating based solely on this study. This intervention may achieve a higher tier when combined with the full body of evidence.
Sample Characteristics
Characteristics of study sample as reported by study author.
-
18% English language learners -
Female: 39%
Male: 61% -
Race Black 19% Other or unknown 4% White 57% -
Ethnicity Hispanic 21% Not Hispanic or Latino 79%
Study Details
Setting
The study was conducted in first-grade classrooms in schools in the United States.
Study sample
From a total of 628 schools in multiple states participating in an i3 scale-up study of Reading Recovery®, 209 schools were randomly selected to participate in this randomized controlled trial. Of those, 158 schools carried out the student-level random assignment process, forming matched pairs of students and randomly assigning one student from each pair to the intervention group and one to the comparison group. In total, 628 students were assigned to the intervention group and 625 students to the comparison group. The analytic sample included only student pairs for whom complete data were available: 866 students in 147 schools, with 433 students in the intervention group and 433 students in the comparison group. In the analytic sample, 61% of the students in the intervention group were male, 17% were English learners, 57% were White, 22% were Hispanic, 18% were African American, and 3% were categorized as other race. In the comparison group, 61% of students were male, 18% were English learners, 56% were White, 20% were Hispanic, 19% were African American, and 5% were categorized as other race.
Intervention Group
Students in the intervention group were pulled out of the classroom for 30 minutes a day for one-on-one sessions with a Reading Recovery® teacher. The sessions included reading familiar books, story composition, assembling stories using cut-up sentences, and previewing and reading new books. Frequent progress monitoring by the Reading Recovery® teacher allowed sessions to be tailored to each student’s needs. Reading Recovery® lessons are discontinued when students demonstrate the ability to consistently read at the average level for their grade—this typically occurs between weeks 12 and 20 of the program. Those who make progress but do not reach average classroom performance after 20 weeks are referred for further evaluation and a plan for future action.
Comparison Group
Students in the comparison group received regular classroom instruction in the reading curriculum; they received no supplemental instruction during the intervention period. After the mid-year administration of the posttest, students in the comparison group were eligible to receive instruction in Reading Recovery® during the remainder of the school year.
Outcome descriptions
The ITBS Total Reading test was used to assess students’ general reading achievement levels. The Total Reading test includes two subtests: Reading Comprehension and Reading Words. For a more detailed description of these outcome measures, see Appendix B. The test was administered mid-year, after the completion of the intervention.
Support for implementation
Reading Recovery® teachers participated in training sessions at designated facilities or at the schools where the teachers worked. In the sessions, teachers were trained to design and implement daily lessons tailored to the needs of the individual student. Teachers also learned to document lesson activities and collect data to track student progress and inform lesson planning. Teacher learning was supported in three main ways: (a) Teachers completed a 1-week summer course that addressed the interpretation and scoring of the Observation Survey of Early Literacy Achievement (the pretest given to students in the evaluation to assess their reading level); (b) Teachers completed a year-long academic course taught by a Reading Recovery® teacher leader, where they attended weekly 3-hour training sessions; and (c) Teachers were observed by and received feedback from their teacher leader.
An indicator of the effect of the intervention, the improvement index can be interpreted as the expected change in percentile rank for an average comparison group student if that student had received the intervention.
For more, please see the WWC Glossary entry for improvement index.
An outcome is the knowledge, skills, and attitudes that are attained as a result of an activity. An outcome measures is an instrument, device, or method that provides data on the outcome.
A finding that is included in the effectiveness rating. Excluded findings may include subgroups and subscales.
The sample on which the analysis was conducted.
The group to which the intervention group is compared, which may include a different intervention, business as usual, or no services.
The timing of the post-intervention outcome measure.
The number of students included in the analysis.
The mean score of students in the intervention group.
The mean score of students in the comparison group.
The WWC considers a finding to be statistically significant if the likelihood that the finding is due to chance alone, rather than a real difference, is less than five percent.
The WWC reviews studies for WWC products, Department of Education grant competitions, and IES performance measures.
The name and version of the document used to guide the review of the study.
The version of the WWC design standards used to guide the review of the study.
The result of the WWC assessment of the study. The rating is based on the strength of evidence of the effectiveness of the intervention. Studies are given a rating of Meets WWC Design Standards without Reservations, Meets WWC Design Standards with Reservations, or >Does Not Meet WWC Design Standards.
A related publication that was reviewed alongside the main study of interest.
Study findings for this report.
Based on the direction, magnitude, and statistical significance of the findings within a domain, the WWC characterizes the findings from a study as one of the following: statistically significant positive effects, substantively important positive effects, indeterminate effects, substantively important negative effects, and statistically significant negative effects. For more, please see the WWC Handbook.
The WWC may review studies for multiple purposes, including different reports and re-reviews using updated standards. Each WWC review of this study is listed in the dropdown. Details on any review may be accessed by making a selection from the drop down list.
Tier 1 Strong indicates strong evidence of effectiveness,
Tier 2 Moderate indicates moderate evidence of effectiveness, and
Tier 3 Promising indicates promising evidence of effectiveness,
as defined in the
non-regulatory guidance for ESSA
and the regulations for ED discretionary grants (EDGAR Part 77).