|
|
The WWC review of interventions for early childhood education addresses child outcomes in six domains: oral language, print knowledge, phonological processing, early reading and writing, cognition, and math. The study included in this report covers four domains: oral language, print knowledge, phonological processing, and math. The findings below present the authors’ estimates and WWC-calculated estimates of the size and the statistical significance of the effects of Bright Beginnings on children.7
Oral Language. The PCER Consortium (2008) analyzed the effectiveness of Bright Beginnings on oral language using the Peabody Picture Vocabulary Test (PPVT-III) and the Test of Language Development (TOLD). The authors report, and the WWC confirms, that differences between the Bright Beginnings and control groups are not statistically significant or substantively important on any of these measures. According to WWC criteria, this study shows no discernible effects on oral language.
Print Knowledge. The PCER Consortium (2008) analyzed the
effectiveness of
Bright Beginnings on the Test of Early Reading
Ability (TERA-3), the Woodcock-Johnson III (WJ-III) Letter-Word Identification subtest,
and the WJ-III Spelling subtest.8 The study authors report a
statistically significant positive effect of Bright Beginnings on the
TERA-3, but according to WWC calculations, after correcting for
multiple comparisons, the effect was not statistically significant.
The study authors report, and the WWC confirms, there was
no statistically significant effect on the WJ-III Letter-Word Identification subtest or on the WJ-III Spelling subtest. Although
the effects are not statistically significant, the PCER Consortium
found, and the WWC confirmed, that the effects on the TERA-3
and on the WJ-III Letter-Word Identification subtest are large
enough to be considered substantively important according to
WWC criteria (that is, at least 0.25). According to WWC criteria,
the study shows potentially positive effects on print knowledge.
Phonological Processing. The PCER Consortium (2008) analyzed the effectiveness of Bright Beginnings on phonological processing using the Preschool Comprehensive Test of Phonological and Print Processing. The authors report, and the WWC confirms, that differences between the Bright Beginnings and control groups are not statistically significant or substantively important on this measure. According to WWC criteria, this study shows no discernible effects on phonological processing.
Math. The PCER Consortium (2008) analyzed the effectiveness of Bright Beginnings on math using the WJ-III Applied Problems subtest, the Composite Score from the Child Math Assessment-Abbreviated, and Shape Composition. The authors report, and the WWC confirms, that differences between the Bright Beginnings and control groups are not statistically significant or substantively important on any of these measures. According to WWC criteria, this study shows no discernible effects on math.
The WWC rates the effects of an intervention in a given outcome domain as positive, potentially positive, mixed, no discernible effects, potentially negative, or negative. The rating of effectiveness takes into account four factors: the quality of the research design, the statistical significance of the findings, the size of the difference between participants in the intervention and the comparison conditions, and the consistency in findings across studies (see the WWC Procedures and Standards Handbook, Appendix E).
7 The level of statistical significance was reported by the study authors or, when necessary, calculated by the WWC to correct for clustering within classrooms
or schools and for multiple comparisons. For an explanation, see the WWC Tutorial on Mismatch. For the formulas the WWC used to calculate the statistical significance, see WWC Procedures and Standards Handbook, Appendix C for clustering and WWC Procedures and Standards Handbook, Appendix D for multiple comparisons. No correction for clustering was needed for the study by the PCER Consortium (2008) because their analysis corrected for clustering by using hierarchical linear modeling (HLM). A correction for multiple comparisons was needed for the PCER Consortium (2008) study, so the significance levels in this report may differ from those reported in the original study.
8 By name, the TERA-3 sounds like it should be captured under the early reading and writing domain; however, the description of the measure identifies constructs that are pertinent to print knowledge, such as knowing the alphabet, understanding print conventions, and environmental print. More detailed explanations of the measures in each domain can be found in Appendices 2.1-2.4.
|Institute of Education Sciences
