The Institute of Education Sciences is conducting a rigorous impact evaluation of two supplemental literacy programs aimed at improving the reading comprehension skills and school performance of struggling ninth-grade students. The present report focuses on the first cohort of ninth grade students who are participating in the study and discusses the impact of the programs – each of which provides an extra class period focused directly on literacy instruction – on reading comprehension and vocabulary skills at the end of the students’ freshman year. The evaluation is a randomized controlled trial (RCT) that compares the outcomes of struggling ninth grade readers randomly assigned to participate or not to participate in one of the two year-long supplemental literacy interventions.
The report on the first cohort of students in this study contains the following key findings:
- On average, across the 34 participating high schools, the supplemental literacy programs improved student reading comprehension test scores. This impact is statistically significant. Despite the improvement in reading comprehension, 76 percent of the students who enrolled in the ERO classes were still reading at two or more years below grade level at the end of ninth grade.
- Although they are not statistically significant, the magnitudes of the impact estimates for each literacy intervention separately are the same as those for the full study sample.
- Impacts on reading comprehension are larger for the 15 schools where (1) the ERO programs began within six weeks of the start of the school year and (2) implementation was classified as moderately or well aligned with the program model, compared with impacts for the 19 schools where at least one of these conditions was not met. The difference in impacts on reading comprehension between these two groups of schools is statistically significant. It is important to note, however, that these two factors did not necessarily cause the differences in impacts and that other factors may be also associated with differences in estimated impacts across schools.
