|
|
The WWC review of interventions for Early Childhood Education addresses student outcomes in six domains: oral language, print knowledge, phonological processing, early reading and writing, cognition, and math. The studies included in this report cover four domains: oral language, print knowledge, phonological processing, and math. The findings below present the authors’ estimates and WWC-calculated estimates of the size and the statistical significance of the effects of Doors to Discovery™ on students.11
Oral Language. The PCER Consortium (2008) analyzed the effectiveness of Doors to Discovery™ on oral language using the Peabody Picture Vocabulary Test-III (PPVT-III) and the Test of Language Development-Primary III (TOLD-P:3) Grammatic Understanding subtest. The authors report, and the WWC confirms, that differences between the Doors to Discovery™ group and the control group are not statistically significant or substantively important on any of these measures. According to WWC criteria, this study shows no discernible effects on oral language.
Christie et al. (2003) analyzed the effectiveness of Doors to Discovery™ on oral language using the PPVT-III. WWC analyses of the Christie et al. (2003) data show a substantively important, but not statistically significant, positive effect of 0.30 when the Doors to Discovery™ group was compared to the control group.12
Print Knowledge. The PCER Consortium (2008) analyzed the effectiveness of Doors to Discovery™ on print knowledge using the Test of Early Reading Ability-III (TERA-3), the Woodcock-Johnson III (WJ-III) Letter-Word Identification subtest, and the WJ-III Spelling subtest. The authors report, and the WWC confirms, that differences between the Doors to Discovery™ group and the control group are not statistically significant or substantively important on any of these measures. According to WWC criteria, this study shows no discernible effects on print knowledge.
Christie et al. (2003) analyzed the effectiveness of Doors to Discovery™ on print knowledge using Get Ready to Read! and the Concepts of Print. WWC analyses of the Christie et al. (2003) data show a substantively important, but not statistically significant, positive effect of 0.74 when Doors to Discovery™ was compared to the control group.
Phonological Processing. The PCER Consortium (2008) analyzed the effectiveness of Doors to Discovery™ on phonological processing using the Preschool Comprehensive Test of Phonological and Print Processing (Pre-CTOPPP) Elision subtest. The authors report, and the WWC confirms, that differences between the Doors to Discovery™ group and the control group are not statistically significant or substantively important on any of these measures. According to WWC criteria, this study shows no discernible effects on phonological processing.
Math. The PCER Consortium (2008) analyzed the effectiveness of Doors to Discovery™ on math using the WJ-III Applied Problems subtest, the Child Math Assessment-Abbreviated (CMA-A), and the Shape Composition task. The authors report, and the WWC confirms, that differences between the Doors to Discovery™ group and the control group are not statistically significant or substantively important on any of these measures. According to WWC criteria, this study shows no discernible effects on math.
The WWC rates the effects of an intervention in a given outcome domain as positive, potentially positive, mixed, no discernible effects, potentially negative, or negative. The rating of effectiveness takes into account four factors: the quality of the research design, the statistical significance of the findings, the size of the difference between participants in the intervention and the comparison conditions, and the consistency in findings across studies (see the WWC Procedures and Standards Handbook, Appendix E).
11 The level of statistical significance was reported by the study authors or, where necessary, calculated by the WWC to correct for clustering within classrooms or schools and for multiple comparisons. For an explanation, see the WWC Tutorial on Mismatch. For the formulas the WWC used to calculate the statistical significance, see WWC Procedures and Standards Handbook, Appendix C for clustering and WWC Procedures and Standards Handbook, Appendix D for multiple comparisons. No correction for clustering was needed for the study by the PCER Consortium (2008) because their analysis corrected for clustering by using hierarchical linear modeling (HLM), but a correction for multiple comparisons was needed, so the significance levels in this report may differ from those reported in the original study. For the study by Christie et al (2003), the WWC excluded the one non-randomly assigned classroom and corrected for clustering, so the significance levels in this report may differ from those reported in the original study.
12 Christie et al. (2003) report a statistical difference for the PPVT-III, but the results are based on a sample of five classrooms. As noted in Appendix A1.2, one of these classrooms was not randomly assigned, and thus excluded from the review.
|Institute of Education Sciences
