Appendix A1.1 Study characteristics: Russell, 2005
| Characteristic | Description |
|---|---|
| Study citation | Russell, J. (2005). An investigation of preschool oral language improvements through Ladders to Literacy. Unpublished master’s thesis, University of New Hampshire, Durham. (62329791). |
| Participants1 | This study was a posttest-only design (no pretest was possible due to delays in study implementation) conducted with children from 12 Head Start classrooms, some of which were included the PCER Consortium (2008) study that is described below. The classrooms were selected in 2002 from a list of prospective study participants and randomly assigned. The researchers first identified four urban full-day classrooms and randomly assigned two to the treatment group and two to the control group. Also, they selected (a) two urban half-day classrooms with high numbers of Spanish-speaking children, (b) two additional urban half-day classrooms, (c) two suburban/rural classrooms from towns with a kindergarten program, and (d) two classrooms from towns with no kindergarten program; from each group, one classroom was randomly assigned to treatment and one to control. The study was conducted in the 2002–03 school year. Study participants were limited to children speaking English as their primary language and not enrolled in a special education program. Although 60 children were eligible for the study, only the 34 children whose parents signed consent forms were included in the sample (18 in the treatment classrooms, 16 in the control classrooms). At baseline, children in the study averaged 4.7 years of age; 65% were male; 12% were Hispanic, 71% were Caucasian, and 6% were African-American; and none of the children were identified as having a disability. |
| Setting | The study took place in 12 Head Start classrooms in southern New Hampshire. |
| Intervention | Ladders to Literacy was implemented as a supplementary curriculum to the Creative Curriculum®. Teachers were trained to implement 18 language and literacy activities (of 50 that were available) across three domains (print/book awareness, metalinguistic awareness, and oral language). Fidelity of implementation was assessed twice during the study year: first in January/February 2003, and again in March/April 2003. For both treatment (Ladders to Literacy plus Creative Curriculum®) and control (Creative Curriculum® alone) classrooms, fidelity for the Creative Curriculum® was assessed using a checklist published by the Creative Curriculum® publishers. For the treatment group, technical assistance was available, if needed. A checklist for Ladders to Literacy was prepared by state Department of Education staff. Across both groups of classrooms, implementation of both curricula was low to moderate, averaging near 50% of the scheduled activities. |
| Comparison | Control classrooms implemented the Creative Curriculum®. Creative Curriculum® classrooms are designed to encourage children’s choices and hands-on learning. Classrooms are set up with “interest areas,” sections of the classrooms with different foci—for example, library area, house corner, and art center—which include relevant toys and objects. Children are encouraged to interact and play in the various interest areas (Russell, 2005). |
| Primary outcomes and measurement | For posttests, oral language is measured through analysis of samples of child speech and calculation of mean length of utterance (MLU) and type token ratio (TTR). For a more detailed description of these outcome measures, see Appendix A2.1. |
| Staff/teacher training | Both treatment and control teachers received at least one day of training in the Creative Curriculum®. Treatment group teachers received an additional two days of training on Ladders to Literacy activities in early fall 2002. |
| 1 This study was conducted during the pilot year of the PCER Consortium (2008) study of Ladders to Literacy. Attrition at both the classroom and student levels was low enough to meet WWC standards for acceptable levels of bias under conservative assumptions. | |
Appendix A1.2 Study characteristics: PCER Consortium, 2008
| Characteristic | Description |
|---|---|
| Study citation | Preschool Curriculum Evaluation Research (PCER) Consortium. (2008). Creative Curriculum with Ladders to Literacy: University of New Hampshire. In Effects of preschool curriculum programs on school readiness (pp. 65–73). Washington, DC: National Center for Education Research, Institute of Education Sciences, U.S. Department of Education. |
| Participants1 | This was a pre-, post-, follow-up study of the effectiveness of the Ladders to Literacy curriculum conducted during the 2003–04 and 2004–05 school years. In 2002–03 (the study’s pilot year), the researchers recruited 12 Head Start classrooms to participate in the study, blocked them in various ways, and randomly assigned them to treatment and control groups from the blocks (described above in Russell, 2005). In the study’s evaluation year (2003–04), 11 of the pilot-year classrooms and nine of the teachers were retained. One control classroom was replaced with another classroom from the same center, and two additional classrooms were randomly assigned to the treatment and control groups. This resulted in a sample of 14 classrooms (seven treatment and seven control). For most of the classrooms, the treatment condition had been in place for a full year when the evaluation year started, and thus parents of children in the new cohort had the opportunity to learn about the treatments and select the one they liked best. After parental consent was obtained, the sample included 123 children at baseline; 105 children were included in the final sample (54 treatment, 51 control). Baseline equivalence between the treatment and control children was established. At baseline, children in the study averaged 4.6 years of age; 44% were male; 31% were Hispanic, 39% were Caucasian, and 11% were African-American. |
| Setting | The study was conducted in Head Start classrooms in New Hampshire. |
| Intervention | Ladders to Literacy was implemented as a supplementary curriculum to the Creative Curriculum®. In this study, the researchers selected 27 of the more than 50 Ladders to Literacy activities to be used in all study classrooms. Teachers were trained to implement 27 language and literacy activities that covered three domains (print/book awareness, metalinguistic awareness, and oral language). Teachers were expected to implement nine activities (three from each of the three major domains) in the months of November and December 2003. Teachers were to add three to six additional activities on a monthly basis from January to May 2004. Researchers used a global fidelity measure to rate the overall fidelity with which the curricula were implemented. On a four-point scale (0 = Not at All to 3 = High), the Ladders to Literacy curriculum was rated in the high-medium range (2.71), whereas the control group curriculum was rated at the medium level (2.0). |
| Comparison | The control group implemented the Creative Curriculum® without Ladders to Literacy. The Creative Curriculum® is a comprehensive curriculum for 3- to 5-year-old children. It addresses four areas of development: social/emotional, physical, cognitive, and language development. Creative Curriculum® requires the physical space of the classroom to be structured into 10 interest areas: blocks, dramatic play, toys and games, art, library, discovery, sand and water, music and movement, cooking, and computers. Time is also allotted for outdoor activities. The 10 interest areas are designed to address curriculum content such as literacy, math, science, social studies, the arts, and technology, as well as process skills such as observing, exploring, and problem solving. Creative Curriculum® includes a developmental checklist that teachers are asked to use in ongoing assessments of child progress. |
| Primary outcomes and measurement | The outcome domains of oral language, print knowledge, phonological processing, and math were assessed with standardized measures. Oral language was assessed with the Peabody Picture Vocabulary Test–Third Edition (PPVT-III) and the Grammatic Understanding subtest from the Test of Oral Language Development–Primary III (TOLD-P:3). Print knowledge was assessed with the Test of Early Reading Ability–Third Edition (TERA-3) and the Woodcock-Johnson III (WJ-III) Letter-Word Identification and Spelling subtests. Phonological processing was assessed with the Elision subtest from the Preschool Comprehensive Test of Phonological and Print Processing (Pre-CTOPPP).2 Math was assessed with the WJ-III Applied Problems subtest, the composite score from the Child Math Assessment–Abbreviated (CMA-A) and the Building Blocks Shape Composition test. Pretesting was done in the fall of the preschool year, and posttesting was done in the spring of the preschool year. Trained research staff administered all assessments, which were conducted in English. For a more detailed description of these outcome measures, see Appendices A2.1–A2.4. |
| Staff/teacher training | All teachers (both treatment and control) received at least one day of training in the Creative Curriculum® from a staff member at Teaching Strategies, Inc. Treatment group teachers received Ladders to Literacy training in September 2003 and ongoing training on a monthly basis throughout the 2003–04 school year. |
|
1 Attrition at both the classroom and student levels was low enough to meet WWC standards for acceptable levels of bias under conservative assumptions. 2 The Comprehensive Test of Phonological Processing (CTOPP) was used in the kindergarten follow-up (reported in Appendix A4.3). |
|
Appendix A2.1 Outcome measures for the oral language domain
| Outcome measure | Description |
|---|---|
| Mean length of utterance (MLU) | All utterances (at least 50 per child) were recorded for each child in the treatment and control groups. This required 15–30 minutes of recording per child. Utterances were transcribed verbatim by staff who were blind to the treatment status of the child. The median 50 utterances were selected from the resulting transcriptions and used to compute MLU. For a given child, MLU is calculated as [number of morphemes]/[number of utterances] based on the entire sample of the child’s speech (as described in Russell, 2005). |
| Type token ratio (TTR) | All utterances (at least 50 per child) were recorded for each child in the treatment and control groups. This required 15–30 minutes of recording per child. Utterances were transcribed verbatim by staff who were blind to the treatment status of the child. The median 50 utterances were selected from the resulting transcriptions and used to compute TTR. For a given child, TTR is calculated as [number of different words in the sample]/[total number of words in the sample] based on the entire sample of the child’s speech (as described in Russell, 2005). |
| Peabody Picture Vocabulary Test–Third Edition (PPVT-III) | A standardized measure of children’s receptive vocabulary for which children show understanding of a spoken word by pointing to a picture that best represents the meaning (as cited in PCER Consortium, 2008). |
| Test of Language Development–Primary III (TOLD-P:3) Grammatic Understanding subtest | A standardized measure of children’s ability to comprehend the meaning of sentences by selecting pictures that most accurately represent the sentence (as cited in PCER Consortium, 2008). |
Appendix A2.2 Outcome measures for the print knowledge domain
| Outcome measure | Description |
|---|---|
| Test of Early Reading Ability–III (TERA-3) Total Score | A standardized measure of children’s developing reading skills with three subtests: Alphabet, Conventions, and Meaning (as cited in PCER Consortium, 2008).1 |
| Woodcock-Johnson III (WJ-III) Letter-Word Identification subtest | A standardized measure of identification of letters and reading of words (as cited in PCER Consortium, 2008). |
| WJ-III Spelling subtest | A standardized measure that assesses children’s prewriting skills, such as drawing lines, tracing, and writing letters (as cited in PCER Consortium, 2008). |
| 1 By name, this measure sounds like it should be captured under the early reading/writing domain; however, the description of the measure identifies constructs that are pertinent to print knowledge, such as knowing the alphabet, understanding print conventions, and environmental print. | |
Appendix A2.3 Outcome measures for the phonological processing domain
| Outcome measure | Description |
|---|---|
| Preschool Comprehensive Test of Phonological and Print Processing (Pre-CTOPPP) Elision subtest | A measure of children’s ability to identify and manipulate sounds in spoken words using word prompts and picture plates for the first nine items and word prompts only for later items (as cited in PCER Consortium, 2008). |
| Comprehensive Test of Phonological Processing (CTOPP) Elision subtest | The CTOPP Elision subtest assesses phonological awareness and is similar to the Pre-CTOPPP Elision subtest but does not include pictures in the administration format (as cited in PCER Consortium, 2008). |
Appendix A2.4 Outcome measures for the math domain
| Outcome measure | Description |
|---|---|
| Woodcock-Johnson III (WJ-III) Applied Problems subtest | A standardized measure of children’s ability to solve numerical and spatial problems, presented verbally with accompanying pictures of objects (as cited in PCER Consortium, 2008). |
| Child Math Assessment–Abbreviated (CMA-A) Composite Score |
The average of four subscales: (1) solving addition and subtraction problems using visible objects, (2) constructing a set of objects equal in number to a given set, (3) recognizing shapes, and (4) copying a pattern using objects that vary in color and identity from the model pattern (as cited in PCER Consortium, 2008). |
| Building Blocks Shape Composition task | Modified for PCER from the Building Blocks assessment tools. Children use blocks to fill in a puzzle and are assessed on whether they fill the puzzle without gaps or hangovers (as cited in PCER Consortium, 2008). |
Appendix A3.1 Summary of study findings included in the rating for the oral language domain1
| Authors’ findings from the study |
||||||||
|---|---|---|---|---|---|---|---|---|
| Mean outcome (standard deviation)2 |
WWC calculations | |||||||
| Outcome measure | Study sample | Sample size (classrooms/children) | Ladders to Literacy3 group | Comparison group | Mean difference4 (Ladders to Literacy – comparison) | Effect size5 | Statistical significance6 (at α = 0.05) |
Improvement index7 |
| Russell, 20058 | ||||||||
| MLU | Preschoolers | 12/34 | 3.36 (0.56) |
3.45 (0.80) |
–0.09 | –0.13 | ns | –5 | TTR | Preschoolers | 12/34 | 0.52 (0.49) |
0.50 (0.06) |
0.02 | 0.04 | ns | +2 |
| Average for oral language (Russell, 2005)9 | –0.04 | ns | –1 | |||||
| PCER Consortium, 20088 | ||||||||
| PPVT-III | Preschoolers | 14/105 | 88.24 (18.03) |
95.43 (14.88) |
–7.19 | –0.38 | ns | –15 |
| TOLD-P:3 Grammatic Understanding subtest | Preschoolers | 14/104 | 8.38 (2.87) |
9.45 (2.61) |
–1.07 | –0.22 | ns | –9 |
| Average for oral language (PCER Consortium, 2008)9 | –0.30 | ns | –12 | |||||
| Domain average for oral language across all studies9 | –0.17 | na | –7 | |||||
|
ns = not statistically significant 1 This appendix reports findings considered for the effectiveness rating and the average improvement indices for the oral language domain. Follow-up findings from PCER Consortium (2008) are not included in these ratings but are reported in Appendix A4.1. |
||||||||
Appendix A3.2 Summary of study findings included in the rating for the print knowledge domain1
| Authors’ findings from the study |
||||||||
|---|---|---|---|---|---|---|---|---|
| Mean outcome (standard deviation)2 |
WWC calculations | |||||||
| Outcome measure | Study sample | Sample size (classrooms/children) | Ladders to Literacy3 group | Comparison group | Mean difference4 (Ladders to Literacy – comparison) | Effect size5 | Statistical significance6 (at α = 0.05) |
Improvement index7 |
| PCER Consortium, 20088 | ||||||||
| TERA-3 | Preschoolers | 14/105 | 82.90 |
85.27 (14.66) |
–2.37 | –0.30 | ns | –12 |
| WJ-III Letter Word Identification subtest |
Preschoolers | 14/105 | 93.81 (13.20) |
97.90 (13.56) |
–4.09 | –0.16 | ns | –6 |
| WJ-III Spelling subtest | Preschoolers | 14/105 | 97.31 (12.13) |
89.96 (15.12) |
7.35 | 0.30 | ns | +12 |
| Domain average for print knowledge9 | –0.05 | na | –2 | |||||
|
ns = not statistically significant 1 This appendix reports findings considered for the effectiveness rating and the average improvement indices for the print knowledge domain. Follow-up findings from PCER Consortium (2008) are not included in these ratings but are reported in Appendix A4.2. |
||||||||
Appendix A3.3 Summary of study findings included in the rating for the phonological processing domain1
| Authors’ findings from the study |
||||||||
|---|---|---|---|---|---|---|---|---|
| Mean outcome (standard deviation)2 |
WWC calculations | |||||||
| Outcome measure | Study sample | Sample size (classrooms/children) | Ladders to Literacy3 group | Comparison group | Mean difference4 (Ladders to Literacy – comparison) | Effect size5 | Statistical significance6 (at α = 0.05) |
Improvement index7 |
| PCER Consortium, 20088 | ||||||||
| Pre-CTOPPP Elision subtest | Preschoolers | 14/105 | 8.55 (4.60) |
9.10 (5.09) |
–0.55 | –0.16 | ns | –6 |
| Domain average for phonological processing9 | –0.16 | na | –6 | |||||
|
ns = not statistically significant 2 The standard deviation across all students in each group shows how dispersed the participants’ outcomes are: a smaller standard deviation on a given measure would indicate that participants had more similar outcomes. 3 For PCER Consortium (2008), each intervention group mean is calculated as the unadjusted control mean plus the covariate-adjusted mean difference. Standard deviations are unadjusted. 4 Positive differences and effect sizes favor the intervention group; negative differences and effect sizes favor the comparison group. 5 For an explanation of the effect size calculation, see WWC Procedures and Standards Handbook, Appendix B. In the case of PCER Consortium (2008), the WWC used the effect sizes reported by the study authors (Cohen’s d based on a repeated measures analysis). 6 Statistical significance is the probability that the difference between groups is a result of chance rather than a real difference between the groups. 7 The improvement index represents the difference between the percentile rank of the average student in the intervention condition and that of the average student in the comparison condition. The improvement index can take on values between –50 and +50, with positive numbers denoting results favorable to the intervention group. 8 The level of statistical significance was reported by the study authors or, when necessary, calculated by the WWC to correct for clustering within classrooms or schools and for multiple comparisons. For the formulas the WWC used to calculate the statistical significance, see WWC Procedures and Standards Handbook, Appendix C for clustering and WWC Procedures and Standards Handbook, Appendix D for multiple comparisons. In the case of PCER Consortium (2008), no corrections for clustering or multiple comparisons were needed because the analysis corrected for clustering using HLM. 9 This row provides the study average, which in this instance is also the domain average. The WWC-computed domain average effect size is a simple average rounded to two decimal places. The domain improvement index is calculated from the average effect size. |
||||||||
Appendix A3.4 Summary of study findings included in the rating for the math domain1
| Authors’ findings from the study |
||||||||
|---|---|---|---|---|---|---|---|---|
| Mean outcome (standard deviation)2 |
WWC calculations | |||||||
| Outcome measure | Study sample | Sample size (classrooms/children) | Ladders to Literacy3 group | Comparison group | Mean difference4 (Ladders to Literacy – comparison) | Effect size5 | Statistical significance6 (at α = 0.05) |
Improvement index7 |
| PCER Consortium, 20088 | ||||||||
| WJ-III Applied Problems subtest | Preschoolers | 14/105 | 93.09 (14.25) |
96.10 (15.39) |
–3.01 | –0.14 | ns | –6 |
| CMA-A Composite Score | Preschoolers | 14/105 | 0.60 (0.23) |
0.56 (0.25) |
0.04 | 0.18 | ns | +7 |
| Building Blocks Shape Composition | Preschoolers | 14/104 | 1.77 |
1.75 (0.98) |
0.02 | 0.02 | ns | +1 |
| Domain average for math9 | 0.02 | na | +1 | |||||
|
ns = not statistically significant 2 The standard deviation across all students in each group shows how dispersed the participants’ outcomes are: a smaller standard deviation on a given measure would indicate that participants had more similar outcomes. 3 For PCER Consortium (2008), each intervention group mean is calculated as the unadjusted control mean plus the covariate-adjusted mean difference. Standard deviations are unadjusted. 4 Positive differences and effect sizes favor the intervention group; negative differences and effect sizes favor the comparison group. 5 For an explanation of the effect size calculation, see WWC Procedures and Standards Handbook, Appendix B. In the case of PCER Consortium (2008), the WWC used the effect sizes reported by the study authors (Cohen’s d based on a repeated measures analysis). 6 Statistical significance is the probability that the difference between groups is a result of chance rather than a real difference between the groups. 7 The improvement index represents the difference between the percentile rank of the average student in the intervention condition and that of the average student in the comparison condition. The improvement index can take on values between –50 and +50, with positive numbers denoting favorable results for the intervention group. 8 The level of statistical significance was reported by the study authors or, when necessary, calculated by the WWC to correct for clustering within classrooms or schools and for multiple comparisons. For the formulas the WWC used to calculate the statistical significance, see WWC Procedures and Standards Handbook, Appendix C for clustering and WWC Procedures and Standards Handbook, Appendix D for multiple comparisons. In the case of PCER Consortium (2008), no corrections for clustering or multiple comparisons were needed because the analysis corrected for clustering using HLM, and no impacts were statistically significant. 9 This row provides the study average, which in this instance is also the domain average. The WWC-computed domain average effect size is a simple average rounded to two decimal places. The domain improvement index is calculated from the average effect size. |
||||||||
Appendix A4.1 Summary of follow-up findings for the oral language domain1
| Authors’ findings from the study |
||||||||
|---|---|---|---|---|---|---|---|---|
| Mean outcome (standard deviation)2 |
WWC calculations | |||||||
| Outcome measure | Study sample | Sample size (classrooms/children) | Ladders to Literacy3 group | Comparison group | Mean difference4 (Ladders to Literacy – comparison) | Effect size5 | Statistical significance6 (at α = 0.05) |
Improvement index7 |
| PCER Consortium, 20088 | ||||||||
| PPVT-III | Kindergarteners | 14/80 | 94.59 (13.33) |
100.23 (9.24) |
–5.64 | –0.30 | ns | –15 |
| TOLD-P:3 Grammatic Understanding subtest |
Kindergarteners | 14/81 | 9.45 (2.58) |
9.74 (1.93) |
–0.29 | –0.06 | ns | –2 |
|
ns = not statistically significant 2 The standard deviation across all students in each group shows how dispersed the participants’ outcomes are: a smaller standard deviation on a given measure would indicate that participants had more similar outcomes. 3 For PCER Consortium (2008), each intervention group mean is calculated as the unadjusted control mean plus the covariate-adjusted mean difference. Standard deviations are unadjusted. 4 Positive differences and effect sizes favor the intervention group; negative differences and effect sizes favor the comparison group. 5 For an explanation of the effect size calculation, see WWC Procedures and Standards Handbook, Appendix B. In the case of PCER Consortium (2008), the WWC used the effect sizes reported by the study authors (Cohen’s d based on a repeated measures analysis). 6 Statistical significance is the probability that the difference between groups is a result of chance rather than a real difference between the groups. 7 The improvement index represents the difference between the percentile rank of the average student in the intervention condition and that of the average student in the comparison condition. The improvement index can take on values between –50 and +50, with positive numbers denoting results favorable to the intervention group. 8 The level of statistical significance was reported by the study authors or, when necessary, calculated by the WWC to correct for clustering within classrooms or schools (corrections for multiple comparisons were not done for findings not included in the overall intervention rating). For the formulas the WWC used to calculate the statistical significance, see WWC Procedures and Standards Handbook, Appendix C. In the case of PCER Consortium (2008), no corrections for multiple comparisons or clustering were needed because the analysis corrected for clustering using HLM, and no impacts were statistically significant. |
||||||||
Appendix A4.2 Summary of follow-up findings for the print knowledge domain1
| Authors’ findings from the study |
||||||||
|---|---|---|---|---|---|---|---|---|
| Mean outcome (standard deviation)2 |
WWC calculations | |||||||
| Outcome measure | Study sample | Sample size (classrooms/children) | Ladders to Literacy3 group | Comparison group | Mean difference4 (Ladders to Literacy – comparison) | Effect size5 | Statistical significance6 (at α = 0.05) |
Improvement index7 |
| PCER Consortium, 20088 | ||||||||
| TERA-3 | Kindergarteners | 14/81 | 87.80 |
92.03 (12.48) |
–4.23 | –0.54 | ns | –21 |
| WJ-III Letter-Word Identification subtest | Kindergarteners | 14/81 | 93.81 (12.75) |
100.60 (13.92) |
–6.79 | –0.27 | ns | –11 |
| WJ-III Spelling subtest | Kindergarteners | 14/81 | 96.40 (16.18) |
98.29 (12.92) |
–1.89 | –0.08 | ns | –3 |
|
ns = not statistically significant 1 This appendix presents follow-up findings for measures that fall in the print knowledge domain. Outcomes from the preschool year were used for rating purposes and are presented in Appendix A3.2. |
||||||||
Appendix A4.3 Summary of follow-up findings for the phonological processing domain1
| Authors’ findings from the study |
||||||||
|---|---|---|---|---|---|---|---|---|
| Mean outcome (standard deviation)2 |
WWC calculations | |||||||
| Outcome measure | Study sample | Sample size (classrooms/children) | Ladders to Literacy3 group | Comparison group | Mean difference4 (Ladders to Literacy – comparison) | Effect size5 | Statistical significance6 (at α = 0.05) |
Improvement index7 |
| PCER Consortium, 20088 | ||||||||
| CTOPP Elision subtest | Kindergarteners | 14/81 | 4.19 (3.66) |
4.60 (4.55) |
–0.41 | –0.10 | ns | –4 |
|
ns = not statistically significant 1 This appendix presents follow-up findings for measures that fall in the phonological processing domain. Outcomes from the preschool year were used for rating purposes and are presented in Appendix A3.3. |
||||||||
Appendix A4.4 Summary of follow-up findings for the math domain1
| Authors’ findings from the study |
||||||||
|---|---|---|---|---|---|---|---|---|
| Mean outcome (standard deviation)2 |
WWC calculations | |||||||
| Outcome measure | Study sample | Sample size (classrooms/children) | Ladders to Literacy3 group | Comparison group | Mean difference4 (Ladders to Literacy – comparison) | Effect size5 | Statistical significance6 (at α = 0.05) |
Improvement index7 |
| PCER Consortium, 20088 | ||||||||
| WJ-III Applied Problems subtest | Kindergarteners | 14/80 | 94.26 (14.70) |
101.26 (9.12) |
–7.00 | –0.33 | ns | –13 |
| CMA-A Composite | Kindergarteners | 14/81 | 0.66 (0.20) |
0.71 (0.13) |
–0.05 | –0.19 | ns | –8 |
| Building Blocks Shape Composition | Kindergarteners | 14/81 | 2.54 (0.78) |
2.63 (0.69) |
–0.09 | –0.10 | ns | –4 |
|
ns = not statistically significant 1 This appendix presents follow-up findings for measures that fall in the math domain. Outcomes from the preschool year were used for rating purposes and are presented in Appendix A3.4. |
||||||||
Appendix A5.1 Ladders to Literacy rating for the oral language domain
The WWC rates an intervention’s effects for a given outcome domain as positive, potentially positive, mixed, no discernible effects, potentially negative, or negative.1
For the outcome domain of oral language, the WWC rated Ladders to Literacy as having potentially negative effects for preschool children. The remaining rating (negative effects) was not considered, as Ladders to Literacy was assigned the highest applicable rating.
| Rating received |
|---|
|
Potentially negative effects: Evidence of a negative effect with no overriding contrary evidence.
|
| Other ratings considered |
|
Positive effects: Strong evidence of a positive effect with no overriding contrary evidence.
|
|
Potentially positive effects: Evidence of a positive effect with no overriding contrary evidence.
|
|
Mixed effects: Evidence of inconsistent effects as demonstrated through either of the following criteria.
|
|
No discernible effects: No affirmative evidence of effects.
|
| 1 For rating purposes, the WWC considers the statistical significance of individual outcomes and the domain-level effect. The WWC also considers the size of the domain-level effect for ratings of potentially positive or potentially negative effects. For a complete description, see the WWC Procedures and Standards Handbook, Appendix E. |
Appendix A5.2 Ladders to Literacy rating for the Print knowledge domain
The WWC rates an intervention’s effects for a given outcome domain as positive, potentially positive, mixed, no discernible effects, potentially negative, or negative.1
For the outcome domain of print knowledge, the WWC rated Ladders to Literacy as having no discernible effects for preschool children.
| Rating received |
|---|
|
No discernible effects: No affirmative evidence of effects.
|
| Other ratings considered |
|
Positive effects: Strong evidence of a positive effect with no overriding contrary evidence.
|
|
Potentially positive effects: Evidence of a positive effect with no overriding contrary evidence.
|
|
Mixed effects: Evidence of inconsistent effects as demonstrated through either of the following criteria.
|
|
Potentially negative effects: Evidence of a negative effect with no overriding contrary evidence.
|
|
Negative effects: Strong evidence of a negative effect with no overriding contrary evidence.
|
| 1 For rating purposes, the WWC considers the statistical significance of individual outcomes and the domain-level effect. The WWC also considers the size of the domain-level effect for ratings of potentially positive or potentially negative effects. For a complete description, see the WWC Procedures and Standards Handbook, Appendix E. |
Appendix A5.3 Ladders to Literacy rating for the phonological processing domain
The WWC rates an intervention’s effects for a given outcome domain as positive, potentially positive, mixed, no discernible effects, potentially negative, or negative.1
For the outcome domain of phonological processing, the WWC rated Ladders to Literacy as having no discernible effects for preschool children.
| Rating received |
|---|
|
No discernible effects: No affirmative evidence of effects.
|
| Other ratings considered |
|
Positive effects: Strong evidence of a positive effect with no overriding contrary evidence.
|
|
Potentially positive effects: Evidence of a positive effect with no overriding contrary evidence.
|
|
Mixed effects: Evidence of inconsistent effects as demonstrated through either of the following criteria.
|
|
Potentially negative effects: Evidence of a negative effect with no overriding contrary evidence.
|
|
Negative effects: Strong evidence of a negative effect with no overriding contrary evidence.
|
| 1 For rating purposes, the WWC considers the statistical significance of individual outcomes and the domain-level effect. The WWC also considers the size of the domain-level effect for ratings of potentially positive or potentially negative effects. For a complete description, see the WWC Procedures and Standards Handbook, Appendix E. |
Appendix A5.4 Ladders to Literacy rating for the math domain
The WWC rates an intervention’s effects for a given outcome domain as positive, potentially positive, mixed, no discernible effects, potentially negative, or negative.1
For the outcome domain of math, the WWC rated Ladders to Literacy as having no discernible effects for preschool children.
| Rating received |
|---|
|
No discernible effects: No affirmative evidence of effects.
|
| Other ratings considered |
|
Positive effects: Strong evidence of a positive effect with no overriding contrary evidence.
|
|
Potentially positive effects: Evidence of a positive effect with no overriding contrary evidence.
|
|
Mixed effects: Evidence of inconsistent effects as demonstrated through either of the following criteria.
|
|
Potentially negative effects: Evidence of a negative effect with no overriding contrary evidence.
|
|
Negative effects: Strong evidence of a negative effect with no overriding contrary evidence.
|
| 1 For rating purposes, the WWC considers the statistical significance of individual outcomes and the domain-level effect. The WWC also considers the size of the domain-level effect for ratings of potentially positive or potentially negative effects. For a complete description, see the WWC Procedures and Standards Handbook, Appendix E. |
Appendix A6 Extent of evidence by domain
| Sample size | ||||
|---|---|---|---|---|
| Outcome domain | Number of studies | Preschool classrooms | Children | Extent of evidence1 |
| Oral language | 2 | 262 | 139 | Medium to large |
| Print knowledge | 1 | 14 | 105 | Small |
| Phonological processing | 1 | 14 | 105 | Small |
| Early reading/writing | na | na | na | na |
Cognition |
na | na | na | na |
| Math | 1 | 14 | 105 | Small |
|
na = not applicable/not studied 1 A rating of “medium to large” requires at least two studies and two schools across studies in one domain, and a total sample size across studies of at least 350 students or 14 classrooms. Otherwise, the rating is “small.” For more details on the extent of evidence categorization, see the WWC Procedures and Standards Handbook, Appendix G. |
||||