Executive Summary

This report presents findings from the third and final year of the Reading First Impact Study (RFIS), a congressionally mandated evaluation of the federal government's $1.0 billion-per-year initiative to help all children read at or above grade level by the end of third grade. The No Child Left Behind Act of 2001 (PL 107-110, Title I, Part B, Subpart 1) established Reading First (RF) and mandated its evaluation. This evaluation is being conducted by Abt Associates and MDRC with collaboration from RMC Research, Rosenblum-Brigham Associates, Westat, Computer Technology Services, DataStar, Field Marketing Incorporated, and Westover Consulting, under the oversight of the U.S. Department of Education, Institute of Education Sciences (IES).

This report examines the impact of Reading First funding on 248 schools in 13 states and includes 17 school districts and one statewide program for a total of 18 sites. The study includes data from three school years: 2004-05, 2005-06 and 2006-07.

The Reading First Impact Study was commissioned to address the following questions:

1. What is the impact of Reading First on student reading achievement?
2. What is the impact of Reading First on classroom instruction?
3. What is the relationship between the degree of implementation of scientifically based reading instruction and student reading achievement?

The primary measure of student reading achievement was the Reading Comprehension subtest from the Stanford Achievement Test—10 (SAT 10), given to students in grades one, two, and three. A secondary measure of student reading achievement in decoding was given to students in first grade. The measure of classroom reading instruction was derived from direct observations of reading instruction, and measures of program implementation were derived from surveys of educational personnel. Findings related to the first two questions are based on results pooled across the study's three years of data collection (2004-05, 2005-06, and 2006-07) for classroom instruction and reading comprehension, results from first grade students in one school year (spring 2007) for decoding, and aspects of program implementation from spring 2007 surveys. Key findings are as follows:

• Reading First produced a positive and statistically significant impact on amount of instructional time spent on the five essential components of reading instruction promoted by the program (phonemic awareness, phonics, vocabulary, fluency, and comprehension) in grades one and two. The impact was equivalent to an effect size of 0.33 standard deviations in grade one and 0.46 standard deviations in grade two.
• Reading First produced positive and statistically significant impacts on multiple practices that are promoted by the program, including professional development in scientifically based reading instruction (SBRI), support from full-time reading coaches, amount of reading instruction, and supports available for struggling readers.
• Reading First did not produce a statistically significant impact on student reading comprehension test scores in grades one, two or three.
• Reading First produced a positive and statistically significant impact on decoding among first grade students tested in one school year (spring 2007). The impact was equivalent to an effect size of 0.17 standard deviations.

Results are also presented from exploratory analyses that examine some hypotheses about factors that might account for the observed patterns of impacts. These analyses are considered exploratory because the study was not designed to provide a rigorous test of these hypotheses, and therefore the results must be considered as suggestive. Across different potential predictors of student outcomes, these exploratory analyses are based on different subgroups of students, schools, grade levels, and/or years of data collection. Key findings from these exploratory analyses are as follows:

• There was no consistent pattern of effects over time in the impact estimates for reading instruction in grade one or in reading comprehension in any grade. There appeared to be a systematic decline in reading instruction impacts in grade two over time.
• There was no relationship between reading comprehension and the number of years a student was exposed to RF.
• There is no statistically significant site-to-site variation in impacts, either by grade or overall, for classroom reading instruction or student reading comprehension.
• There is a positive association between time spent on the five essential components of reading instruction promoted by the program and reading comprehension measured by the SAT 10, but these findings are sensitive to both model specification and the sample used to estimate the relationship.

Top