|
|
Effectiveness
Findings
The WWC review of interventions for beginning reading addresses student outcomes in four domains: alphabetics, fluency, comprehension, and general reading achievement. 6 The studies included here cover outcomes in alphabetics and comprehension. Within alphabetics, results for four constructs are reported: phonological awareness, print awareness, letter knowledge, and phonics. The findings below present the authors' estimates and WWC-calculated estimates of the size and statistical significance of the effects of Voyager Universal Literacy System® on students. 7
Alphabetics
Phonological awareness. Frechtling, Zhang, and Silverstein (2006) reported positive, but not statistically significant, effects on the four phonological awareness measures (Comprehensive Test of Phonological Processing (CTOPP) Elision, Blending Words, Blending Nonwords, and Segmentation subtests).
Hecht (2003) examined effects for three phonological awareness measures (Blending, CTOPP Elision, and CTOPP Segmentation) and found a positive and statistically significant effect on the CTOPP Segmentation subtest. None of these effects were statistically significant according to the WWC analysis.
Letter Knowledge. Frechtling, Zhang, and Silverstein (2006) found a positive, but not statistically significant effect on the Dynamic Indicators of Basic Early Literacy Skills (DIBELS) Letter Naming Fluency subtest.
Hecht (2003) reported a positive and statistically significant effect on using a researcher-designed measure of letter naming fluency. According to WWC calculations, the effect was not statistically significant.
Print Awareness. Hecht (2003) reported a negative, but not statistically significant, effect on the Concepts about Print subtest.
Phonics. Frechtling, Zhang, & Silverstein (2006) reported positive, but not statistically significant, effects for the two Woodcock Reading Mastery (WRMT) subtests: Word Identification and Word Attack.
Hecht (2003) reported a statistically significant positive effect for the Letter Sounds test. However, this outcome was not statistically significant according to the WWC analysis. The study found negative effects on the DIBELS Nonsense Word Fluency and Woodcock Word Identification subtests and a positive effect on the Woodcock Reading Mastery Test-Revised (WRMT-R) Word Attack subtest, but none of the effects were statistically significant.
Across all constructs in the alphabetics domain, the average effect size in Frechtling, Zhang, & Silverstein (2006) was positive and large enough to be considered substantively important according to the WWC criteria (that is, at least 0.25). The average effect size for Hecht (2003) was positive, but not large enough to be considered substantively important.
Comprehension
Hecht (2003) reported a negative and statistically significant effect on the Stanford-Binet Intelligence Expressive Vocabulary subtest. This outcome was not statistically significant according to the WWC analysis but the effect size was large enough to be substantively important (that is, smaller than -0.25).
Rating of effectiveness
The WWC rates the effects of an intervention in a given outcome domain as positive, potentially positive, mixed, no discernible effects, potentially negative, or negative. The rating of effectiveness takes into account four factors: the quality of the research design, the statistical significance of the findings,8 the size of the difference between participants in the intervention and the comparison conditions, and the consistency in findings across studies (see the WWC Intervention Rating Scheme).
7 The level of statistical significance was reported by the study authors or, where necessary, calculated by the WWC to correct for clustering within classrooms or schools and for multiple comparisons. For an explanation about the clustering correction, see the WWC Tutorial on Mismatch. See Technical Details of WWC-Conducted Computations for the formulas the WWC used to calculate statistical significance. In the case of all studies of the Voyager Universal Literacy System®, corrections for clustering and multiple comparisons were needed, so the significance levels differ from those reported in the original studies.
8 The level of statistical significance was reported by the study authors or, where necessary, calculated by the WWC to correct for clustering within classrooms or schools and for multiple comparisons. For an explanation, see the WWC Tutorial on Mismatch. See the Technical Details of WWC-Conducted Computations for the formulas the WWC used to calculate the statistical significance. In the case of Voyager Universal Literacy System®, corrections for clustering and multiple comparisons were needed.
