Exploratory Analyses of Variations in Impacts and Relationships among Outcomes
This report also presents results from exploratory analyses that examine some hypotheses about factors
that might account for the pattern of observed impacts presented above. These exploratory analyses are
based on analyses of subgroups of students, schools, grade levels, and/or years of data collection. The
information is provided as possible avenues for further exploration or for improving Reading First or
programs like Reading First. However, the study was not designed to provide a rigorous test of these
hypotheses, and therefore the results are only suggestive. Findings from these exploratory analyses
include the following:
- Data collected during three school years (2004-05, 2005-06 and 2006-07) were used to
examine variation over time in program impacts. No consistent pattern of differential impacts
over time was established.
- No relationship was found between the number of years a student was exposed to RF and
student reading achievement.
- There was no statistically significant variation in impacts across sites in the study, either by
grade or overall, for reading instruction or for reading comprehension.
- Correlational analyses, which are outside the causal framework of the main impact analyses
presented in the report, indicate a positive and statistically significant association between
time spent on the five essential components of reading instruction promoted by the program
and students' reading comprehension. A one-minute increase in time devoted to instruction in
the five dimensions per daily reading block was associated with a 0.07 point increase in
scaled score points in first grade, and a 0.06 point increase in second grade. This relationship
does not hold for models that include other potential mediators of student achievement.
However, due to data limitations, these latter models could only be run on a subset of the
data; thus, we do not know whether the differences in the findings across models are due to
changes in the sample or changes in the model specification itself.