|
|
Adolescent Literacy
Effectiveness
Findings
The WWC review of interventions for AL addresses student outcomes in four domains: alphabetics, reading fluency, comprehension, and general literacy achievement. The studies included in this report cover two domains: comprehension and general literacy achievement. Comprehension includes two constructs: reading comprehension and vocabulary development. The findings below present the authors’ estimates and WWC-calculated estimates of the size and the statistical significance of the effects of READ 180 on adolescent learners.13
Comprehension. Six studies reviewed findings in the comprehension domain. Haslam, White, and Klinge (2006) reported a statistically significant effect of READ 180 on the Texas Assessment of Knowledge and Skills Reading Test. The WWC found that the effect was not statistically significant or large enough to be considered substantively important according to WWC criteria. Interactive Inc. (2002) reported statistically significant effects of READ 180 on the Stanford Achievement Test for the Columbus and Dallas school districts but did not report a statistically significant effect of READ 180 in the Houston school district. According to WWC calculations, the Columbus and Dallas effects were not statistically significant, but the average effect size across the three sites was large enough to be considered substantively important (that is, an effect size of at least 0.25).14 Lang et al. (2008) reported, and the WWC confirmed, a statistically significant effect of READ 180 on the Florida Comprehensive Assessment Test for moderate-risk students. For high-risk students, the effect was not statistically significant or large enough to be considered substantively important according to WWC criteria. White, Haslam, and Hewes (2006) found, and the WWC confirmed, a statistically significant effect of READ 180 on the Reading Comprehension subtest of the Stanford Achievement Test. White, Williams, and Haslem (2005) did not find a statistically significant effect of READ 180 on the New York State English Language Arts Test for grades 4 or 8 or on the CTB/McGraw-Hill Reading Test for grade 6. The WWC-calculated average effect across these three grades and two tests was not statistically significant or large enough to be considered substantively important according to WWC criteria. Woods (2007) did not find a statistically significant effect of READ 180 on the Degrees of Reading Power test. The effect also was not statistically significant or large enough to be considered substantively important according to WWC criteria.
For the comprehension domain, two studies showed statistically significant positive effects, one study showed substantively important positive effects, and three studies showed indeterminate effects.
General literacy achievement. Two studies reviewed findings in the general literacy achievement domain. Scholastic Research (2008) reported, and the WWC confirmed, a statistically significant effect of READ 180 on the English Language Arts subtest of the California Standards Test. White, Haslam, and Hewes (2006) reported a statistically significant positive effect of READ 180 on the TerraNova Reading Test among cohort 2 and cohort 3 students. The WWC confirmed the statistically significant positive effect for cohort 2 students but found that the effect for cohort 3 students was not statistically significant or large enough to be considered substantively important according to the WWC criteria.
For the general literacy achievement domain, two studies showed statistically significant positive effects.
Rating of effectiveness
The WWC rates the effects of an intervention in a given outcome domain as positive, potentially positive, mixed, no discernible effects, potentially negative, or negative. The rating of effectiveness takes into account four factors: the quality of the research design, the statistical significance of the findings, the size of the difference between participants in the intervention and the comparison conditions, and the consistency in findings across studies (see the WWC Procedures and Standards Handbook, Appendix E).
13 The level of statistical significance was reported by the study authors or, where necessary, calculated by the WWC to correct for clustering within classrooms or schools and for multiple comparisons. For an explanation, see the WWC Tutorial on Mismatch. For the formulas the WWC used to calculate the statistical significance, see WWC Procedures and Standards Handbook, Appendix C for clustering and WWC Procedures and Standards Handbook, Appendix D for multiple comparisons. In the cases of Haslam, White, and Klinge (2006); Scholastic Research (2008); White, Haslam, and Hewes (2006); and Woods (2007), no corrections for clustering or multiple comparisons were needed. In the cases of Lang et al. (2008) and White, Williams, and Haslem (2005), corrections for multiple comparisons were needed, so the significance levels may differ from those reported in the original study. In the case of Interactive Inc. (2002), corrections for clustering and multiple comparisons were needed, so the significance levels may differ from those reported in the original study.
14 The WWC computes an average effect size as a simple average of the effect sizes across all individual findings within the study domain.