| Exhibit ES.3. Estimated Impacts on Reading Comprehension, Instruction, and Percentage of Students Engaged with Print: 2005, 2006, and 2007 (pooled) |
|---|
| Actual Mean with Reading First |
Estimated Mean without Reading First |
Impact | Effect Size of Impact |
Statistical Significance of Impact (p-value) |
|
|---|---|---|---|---|---|
| Instruction | |||||
| Number of minutes of instruction in the five components combined | |||||
| Grade 1 | 59.23 | 52.31 | 6.92* | 0.33* | (0.005) |
| Grade 2 | 59.08 | 49.30 | 9.79* | 0.46* | (<0.001) |
| Percentage of intervals in five components with Highly Explicit Instruction | |||||
| Grade 1 | 29.39 | 26.10 | 3.29* | 0.18* | (0.018) |
| Grade 2 | 30.95 | 27.95 | 3.00* | 0.16* | (0.040) |
| Percentage of intervals in five components with High Quality Student Practice | |||||
| Grade 1 | 18.44 | 17.61 | 0.82 | 0.05 | (0.513) |
| Grade 2 | 17.82 | 14.88 | 2.94* | 0.16* | (0.019) |
| Reading Comprehension | |||||
| Scaled Score | |||||
| Grade 1 | 543.8 | 539.1 | 4.7 | 0.10 | (0.083) |
| Grade 2 | 584.4 | 582.8 | 1.7 | 0.04 | (0.462) |
| Grade 3 | 609.1 | 608.8 | 0.3 | 0.01 | (0.887) |
| Percent Reading At or Above Grade Level | |||||
| Grade 1 | 46.0 | 41.8 | 4.2 | -- | (0.104) |
| Grade 2 | 38.9 | 37.3 | 1.6 | -- | (0.504) |
| Grade 3 | 38.7 | 38.8 | -0.1 | -- | (0.973) |
| Percentage of Students Engaged with Print | |||||
| Grade 1 | 47.84 | 42.52 | 5.33 | 0.18 | (0.070) |
| Grade 2 | 50.53 | 55.27 | -4.75 | -0.17 | (0.104) |
| NOTES:
The complete Reading First Impact Study (RFIS) sample includes 248 schools from 18 sites (17 school districts and 1 state) located in 13 states. 125 schools are Reading First schools and 123 are non-Reading First schools. For grade 2 in 2006, one non-RF school could not be included in the analysis because test score data were not available. For grade 3 in 2007, one RF school could not be included in the analysis because test score data were not available. Impact estimates are statistically adjusted (e.g., take each school's rating, site-specific funding cut-point, and other covariates into account) to reflect the regression discontinuity design of the study. Values in the "Actual Mean with Reading First" column are actual, unadjusted values for Reading First schools; values in the "Estimated Mean without Reading First" column represent the best estimates of what would have happened in RF schools absent RF funding and are calculated by subtracting the impact estimates from the RF schools' actual mean values. A two-tailed test of significance was used; statistically significant findings at the p≤.05 level are indicated by *. 1 Except for STEP, which is pooled across 2006 and 2007 school years only. EXHIBIT READS: The observed mean amount of time spent per daily reading block in instruction in the five components combined for first grade classrooms with Reading First was 59.23 minutes. The estimated mean amount of time without Reading First was 52.31 minutes. The impact of Reading First on the amount of time spent in instruction in the five components combined was 6.92 (or 0.33 standard deviations), which was statistically significant (p=.005). SOURCES: RFIS SAT 10 administrations in the spring of 2005, 2006, and 2007, as well as from state/district education agencies in those sites that already used the SAT 10 for their standardized testing (i.e., FL, KS, MD, OR); RFIS Instructional Practice in Reading Inventory, spring 2005, fall 2005, spring 2006, fall 2006, and spring 2007; RFIS Student Time-on-Task and Engagement with Print, fall 2005, spring 2006, fall 2006, and spring 2007. |
|||||