Effects of a Pre-Kindergarten Mathematics Intervention: A Randomized Experiment [Pre-K Mathematics and DLM Express math software vs. business as usual (Creative Curriculum or HighScope)]
Klein, Alice; Starkey, Prentice; Clements, Douglas; Sarama, Julie; Iyer, Roopa (2008). Journal of Research on Educational Effectiveness, v1 n3 p155-178 2008. Retrieved from: https://eric.ed.gov/?id=EJ873866
-
examining278Students, gradePK
Pre-K Mathematics Intervention Report - Preparing Young Children for School
Review Details
Reviewed: July 2023
- Randomized Controlled Trial
- Meets WWC standards without reservations because it is a cluster randomized controlled trial with low cluster-level attrition and individual-level non-response.
This review may not reflect the full body of research evidence for this intervention.
Evidence Tier rating based solely on this study. This intervention may achieve a higher tier when combined with the full body of evidence.
Please see the WWC summary of evidence for Pre-K Mathematics.
Findings
|
Outcome measure |
Comparison | Period | Sample |
Intervention mean |
Comparison mean |
Significant? |
Improvement index |
Evidence tier |
|---|---|---|---|---|---|---|---|---|
|
Test of Language Development: Grammatical Understanding Subtest |
Pre-K Mathematics vs. Business as usual |
0 Days |
Full sample;
|
9.51 |
9.25 |
No |
-- | |
|
Peabody Picture Vocabulary Test-III |
Pre-K Mathematics vs. Business as usual |
0 Days |
Full sample;
|
94.99 |
93.92 |
No |
-- |
|
Outcome measure |
Comparison | Period | Sample |
Intervention mean |
Comparison mean |
Significant? |
Improvement index |
Evidence tier |
|---|---|---|---|---|---|---|---|---|
|
Researcher-Developed: Child Math Assessment (CMA) |
Pre-K Mathematics vs. Business as usual |
0 Days |
Full sample;
|
0.54 |
0.47 |
Yes |
|
|
|
Outcome measure |
Comparison | Period | Sample |
Intervention mean |
Comparison mean |
Significant? |
Improvement index |
Evidence tier |
|---|---|---|---|---|---|---|---|---|
|
Woodcock-Johnson III (W-J III) Spelling Subtest |
Pre-K Mathematics vs. Business as usual |
0 Days |
Full sample;
|
95.39 |
91.55 |
No |
-- | |
|
Woodcock-Johnson III Tests of Achievement (WJ-III) Letter-Word Identification Subtest |
Pre-K Mathematics vs. Business as usual |
0 Days |
Full sample;
|
102.54 |
100.89 |
No |
-- | |
|
Test of Early Reading Ability - 3rd Edition (TERA-3) |
Pre-K Mathematics vs. Business as usual |
0 Days |
Full sample;
|
90.62 |
89.88 |
No |
-- |
|
Outcome measure |
Comparison | Period | Sample |
Intervention mean |
Comparison mean |
Significant? |
Improvement index |
Evidence tier |
|---|---|---|---|---|---|---|---|---|
|
Social Skills Rating System (SSRS): Teacher Report |
Pre-K Mathematics vs. Business as usual |
0 Days |
Full sample;
|
110.81 |
108.65 |
No |
-- |
Evidence Tier rating based solely on this study. This intervention may achieve a higher tier when combined with the full body of evidence.
Sample Characteristics
Characteristics of study sample as reported by study author.
-
Female: 48%
Male: 52% -
- B
- A
- C
- D
- E
- F
- G
- I
- H
- J
- K
- L
- P
- M
- N
- O
- Q
- R
- S
- V
- U
- T
- W
- X
- Z
- Y
- a
- h
- i
- b
- d
- e
- f
- c
- g
- j
- k
- l
- m
- n
- o
- p
- q
- r
- s
- t
- u
- v
- x
- w
- y
California, New York
-
Race Asian 4% Black 53% Other or unknown 21% White 22% -
Ethnicity Hispanic 22% Other or unknown 78% -
Eligible for Free and Reduced Price Lunch Other or unknown 100%
Study Details
Setting
The study took place in 40 Head Start and state-funded preschool classrooms within 6 centers located in the San Francisco Bay Area in California and in the Buffalo, New York area. Of the four San Francisco centers, two were Head Start programs and two were state-funded preschool programs. Of the two Buffalo centers, one was a Head Start program and two were state-funded preschool programs. Ten Head Start classes and 10 state-funded preschool classrooms were included from each state. Classrooms implementing the intervention were not located in the same buildings as classrooms participating in the comparison group.
Study sample
The mean age of the student sample was 4.4 years (range 3.8-4.9 years). All children qualified for kindergarten in the next year, were considered normally developing, and were from low-income families. The racial/ethnic composition of the analytic sample was 53% African American, 22% White, 22% Hispanic, 4% Asian American, and 4% interracial/other. Approximately 48% of the sample was female. The 40 teachers included in the study were 38% White, 33% African American, 13% Hispanic, 10% Asian American, and 5% interracial/other. Teachers had an average of 12.4 years of experience teaching preschool, and 73% of teachers had at least a bachelor's degree.
Intervention Group
Teachers in the intervention group used the Pre-K Mathematics curriculum. The intervention consisted of small-group classroom, home, and computer-based components for mathematics enrichment. The curriculum included units on (a) Counting and Numbers, (b) Understanding Arithmetic Operations, (c) Spatial Sense and Geometry, (d) Patterns, (e) Measurement and Data, and (f) Logical Reasoning. Teachers introduced roughly one activity per week for small groups of students, ranging from 4 to 6 students. There were 29 small-group activities included in the intervention. Each activity was conducted twice in each week for 20 minutes, for a total of 40 minutes per week. Across the 29 activities, there was a total of 58 20-minute small-group activities over the course of the year. The small group activities included suggestions for scaffolding if children struggled with any part of the activity as well as extensions for students that completed the activity relatively easily. Teachers used Assessment Record Sheets to track the progress of each student on each activity. In addition, teachers sent home 18 activities accompanying the in-class activities once every 1-2 weeks. Home activities included manipulatives for parents to use with children, a brief narrative describing the activity and stating its purpose, and picture strips depicting how to conduct the activity. Finally, students completed computer-based activities at the centers using DLM Express, a software program offering 27 supplementary computer-based activities and a math learning center.
Comparison Group
The comparison condition received classes as usual, through Creative Curriculum, High Scope, Montessori, or locally developed curricula. The study team told teachers not to alter their classroom practices or curriculum during the study period.
Support for implementation
Intervention teachers participated in two 4-day training sessions in Pre-K Mathematics. The intervention teachers participated in two workshops plus on-site training twice per month in their classrooms for an average of 15 on-site training sessions per teacher. Teachers were also provided with feedback from observations of the implementation of small-group activities and the use of computer-based mathematics activities.
Additional Sources
In the case of multiple manuscripts that report on one study, the WWC selects one manuscript as the primary citation and lists other manuscripts that describe the study as additional sources.
-
Starkey, P., & Klein, A. (2005). A longitudinal study of the effects of a pre-kindergarten mathematics curriculum on low-income children's mathematical knowledge. Berkeley, CA: University of California.
-
Preschool Curriculum Evaluation ResearchPreschool Curriculum Evaluation Research (2008). Pre-K Mathematics supplemented with DLM Early Childhood Express Math software: University of California, Berkeley and University at Buffalo, State University of New York. In Effects of preschool curriculum programs on school readiness (pp. 131–142). Washington, DC: National Center for Education Research, Institute of Education Sciences, U.S. Department of Education.
Practice Guide
Review Details
Reviewed: July 2023
- Practice Guide (findings for Pre-K Mathematics)
- Randomized Controlled Trial
- Meets WWC standards without reservations because it is a cluster randomized controlled trial with low cluster-level attrition and individual-level non-response.
This review may not reflect the full body of research evidence for this intervention.
Evidence Tier rating based solely on this study. This intervention may achieve a higher tier when combined with the full body of evidence.
Findings
|
Outcome measure |
Comparison | Period | Sample |
Intervention mean |
Comparison mean |
Significant? |
Improvement index |
Evidence tier |
|---|---|---|---|---|---|---|---|---|
|
Test of Language Development: Grammatical Understanding Subtest |
Pre-K Mathematics vs. Business as usual |
0 Days |
Full sample;
|
9.51 |
9.25 |
No |
-- | |
|
Peabody Picture Vocabulary Test-III |
Pre-K Mathematics vs. Business as usual |
0 Days |
Full sample;
|
94.99 |
93.92 |
No |
-- |
|
Outcome measure |
Comparison | Period | Sample |
Intervention mean |
Comparison mean |
Significant? |
Improvement index |
Evidence tier |
|---|---|---|---|---|---|---|---|---|
|
Child Math Assessment (CMA) |
Pre-K Mathematics vs. Business as usual |
0 Days |
Full sample;
|
0.54 |
0.47 |
Yes |
|
|
|
Outcome measure |
Comparison | Period | Sample |
Intervention mean |
Comparison mean |
Significant? |
Improvement index |
Evidence tier |
|---|---|---|---|---|---|---|---|---|
|
Woodcock-Johnson III (W-J III) Spelling subtest |
Pre-K Mathematics vs. Business as usual |
0 Days |
Full sample;
|
95.39 |
91.55 |
No |
-- | |
|
Woodcock-Johnson III Tests of Achievement (WJ-III) Letter-Word Identification Subtest |
Pre-K Mathematics vs. Business as usual |
0 Days |
Full sample;
|
102.54 |
100.89 |
No |
-- | |
|
Test of Early Reading Ability - 3rd Edition (TERA-3) |
Pre-K Mathematics vs. Business as usual |
0 Days |
Full sample;
|
90.62 |
89.88 |
No |
-- |
|
Outcome measure |
Comparison | Period | Sample |
Intervention mean |
Comparison mean |
Significant? |
Improvement index |
Evidence tier |
|---|---|---|---|---|---|---|---|---|
|
Social Skills Rating System (Klein et al 2008) |
Pre-K Mathematics vs. Business as usual |
0 Days |
Full sample;
|
110.81 |
108.65 |
No |
-- |
Evidence Tier rating based solely on this study. This intervention may achieve a higher tier when combined with the full body of evidence.
Sample Characteristics
Characteristics of study sample as reported by study author.
-
Female: 48%
Male: 52% -
- B
- A
- C
- D
- E
- F
- G
- I
- H
- J
- K
- L
- P
- M
- N
- O
- Q
- R
- S
- V
- U
- T
- W
- X
- Z
- Y
- a
- h
- i
- b
- d
- e
- f
- c
- g
- j
- k
- l
- m
- n
- o
- p
- q
- r
- s
- t
- u
- v
- x
- w
- y
California, New York
-
Race Asian 4% Black 53% Other or unknown 21% White 22% -
Ethnicity Hispanic 22% Other or unknown 78% -
Eligible for Free and Reduced Price Lunch Other or unknown 100%
Study Details
Setting
The study took place in 40 Head Start and state-funded preschool classrooms within 6 centers located in the San Francisco Bay Area in California and in the Buffalo, New York area. Of the four San Francisco centers, two were Head Start programs and two were state-funded preschool programs. Of the two Buffalo centers, one was a Head Start program and two were state-funded preschool programs. Ten Head Start classes and 10 state-funded preschool classrooms were included from each state. Classrooms implementing the intervention were not located in the same buildings as classrooms participating in the comparison group.
Study sample
The mean age of the student sample was 4.4 years (range 3.8-4.9 years). All children qualified for kindergarten in the next year, were considered normally developing, and were from low-income families. The racial/ethnic composition of the analytic sample was 53% African American, 22% White, 22% Hispanic, 4% Asian American, and 4% interracial/other. Approximately 48% of the sample was female. The 40 teachers included in the study were 38% White, 33% African American, 13% Hispanic, 10% Asian American, and 5% interracial/other. Teachers had an average of 12.4 years of experience teaching preschool, and 73% of teachers had at least a bachelor's degree.
Intervention Group
Teachers in the intervention group used the Pre-K Mathematics curriculum. The intervention consisted of small-group classroom, home, and computer-based components for mathematics enrichment. The curriculum included units on (a) Counting and Numbers, (b) Understanding Arithmetic Operations, (c) Spatial Sense and Geometry, (d) Patterns, (e) Measurement and Data, and (f) Logical Reasoning. Teachers introduced roughly one activity per week for small groups of students, ranging from 4 to 6 students. There were 29 small-group activities included in the intervention. Each activity was conducted twice in each week for 20 minutes, for a total of 40 minutes per week. Across the 29 activities, there was a total of 58 20-minute small-group activities over the course of the year. The small group activities included suggestions for scaffolding if children struggled with any part of the activity as well as extensions for students that completed the activity relatively easily. Teachers used Assessment Record Sheets to track the progress of each student on each activity. In addition, teachers sent home 18 activities accompanying the in-class activities once every 1-2 weeks. Home activities included manipulatives for parents to use with children, a brief narrative describing the activity and stating its purpose, and picture strips depicting how to conduct the activity. Finally, students completed computer-based activities at the centers using DLM Express, a software program offering 27 supplementary computer-based activities and a math learning center.
Comparison Group
The comparison condition received classes as usual, through Creative Curriculum, High Scope, Montessori, or locally developed curricula. The study team told teachers not to alter their classroom practices or curriculum during the study period.
Support for implementation
Intervention teachers participated in two 4-day training sessions in Pre-K Mathematics. The intervention teachers participated in two workshops plus on-site training twice per month in their classrooms for an average of 15 on-site training sessions per teacher. Teachers were also provided with feedback from observations of the implementation of small-group activities and the use of computer-based mathematics activities.
Additional Sources
In the case of multiple manuscripts that report on one study, the WWC selects one manuscript as the primary citation and lists other manuscripts that describe the study as additional sources.
-
Starkey, P., & Klein, A. (2005). A longitudinal study of the effects of a pre-kindergarten mathematics curriculum on low-income children's mathematical knowledge. Berkeley, CA: University of California.
-
Preschool Curriculum Evaluation ResearchPreschool Curriculum Evaluation Research (2008). Pre-K Mathematics supplemented with DLM Early Childhood Express Math software: University of California, Berkeley and University at Buffalo, State University of New York. In Effects of preschool curriculum programs on school readiness (pp. 131–142). Washington, DC: National Center for Education Research, Institute of Education Sciences, U.S. Department of Education.
Pre-K Mathematics Intervention Report - Early Childhood Education
Review Details
Reviewed: December 2013
- Randomized Controlled Trial
- Meets WWC standards without reservations
This review may not reflect the full body of research evidence for this intervention.
Evidence Tier rating based solely on this study. This intervention may achieve a higher tier when combined with the full body of evidence.
Please see the WWC summary of evidence for Pre-K Mathematics.
Findings
|
Outcome measure |
Comparison | Period | Sample |
Intervention mean |
Comparison mean |
Significant? |
Improvement index |
Evidence tier |
|---|---|---|---|---|---|---|---|---|
|
Child Math Assessment (CMA) |
Pre-K Mathematics vs. business as usual |
Posttest |
Preschool children;
|
0.55 |
0.47 |
Yes |
|
|
Evidence Tier rating based solely on this study. This intervention may achieve a higher tier when combined with the full body of evidence.
Sample Characteristics
Characteristics of study sample as reported by study author.
-
Urban
-
- B
- A
- C
- D
- E
- F
- G
- I
- H
- J
- K
- L
- P
- M
- N
- O
- Q
- R
- S
- V
- U
- T
- W
- X
- Z
- Y
- a
- h
- i
- b
- d
- e
- f
- c
- g
- j
- k
- l
- m
- n
- o
- p
- q
- r
- s
- t
- u
- v
- x
- w
- y
California, New York
-
Race Asian 4% Black 53% Other or unknown 4% White 22% -
Ethnicity Hispanic 22% Not Hispanic or Latino 78%
Study Details
Setting
The study took place in 40 Head Start and state-funded preschool classrooms in California and New York. The classrooms were from four programs in California and two programs in New York.
Study sample
Twenty Head Start and 20 state-funded preschool classrooms were randomly assigned within program type either to an intervention group receiving Pre-K Mathematics with DLM Early Childhood Express Math or to a business-as-usual comparison group. Teachers volunteered to participate in the study, which took place during the 2002–03 school year. Of the 40 participating in the study, 33 were included in the PCER Consortium (2008) study that is described in Appendix A.2. The study began with 316 children from low-income families, combined across states. During the study, 38 children left, resulting in an analysis sample of 278 children. The mean age of the children in cohort 1 was 4.4 years. Fifty-three percent of the children were African American, 22% were Hispanic, 22% were Caucasian, 4% were Asian American, and 4% were interracial or another ethnicity. Forty-eight percent of the children were female.
Intervention Group
Teachers implemented the Pre-K Mathematics curriculum classroom activities with small groups of four to six children in twice-a-week, 20-minute sessions for 29 weeks. Each week involved a new math activity. The small-group sessions included activities from the seven units of the curriculum: (a) counting and number, (b) understanding arithmetic operations (fall unit), (c) spatial sense and geometry, (d) patterns, (e) understanding arithmetic operations (spring unit), (f) measurement and data, and (g) logical reasoning. In addition, teachers supplemented the Pre-K Mathematics curriculum activities with two other instructional activities: (a) 27 computer activities based on the DLM Early Childhood Express Math software, and (b) mathematics learning centers, which included materials from the small-group activities and additional mathematics materials from the classroom. Home activity materials parallel to the classroom activities were sent home every 1 to 2 weeks for parents to use with their children. Teachers tracked children’s progress using a Math Mastery Form, and intervention fidelity data were collected using the Fidelity of Implementation Record Sheet developed by the first two authors of the study. In addition, teachers used a pre-existing general curriculum in their classrooms (curricula included The Creative Curriculum®, High/Scope, Montessori, or specialized literacy curricula and curricula developed by local teachers and school districts).
Comparison Group
Teachers in the comparison group classrooms implemented the curriculum as business-as-usual in their programs. The curricula used in the comparison group classrooms included The Creative Curriculum®, High/Scope, Montessori, or specialized literacy curricula and curricula developed by local teachers and school districts.
Outcome descriptions
The study measured intervention and comparison group children’s mathematical knowledge with the researcher-developed CMA. All the children in the study were assessed at pretest before implementation of the intervention began in intervention classrooms, and at posttest after intervention classrooms completed implementation of the program. For a more detailed description of the CMA, see Appendix B.
Support for implementation
Teachers participated in a 4-day workshop at the beginning of the school year that focused on implementing the first three units of the curriculum and another 4-day workshop at mid-year that focused on implementing units four through seven. In addition, teachers were provided on-site training approximately twice a month and implementation fidelity checks once or twice each month.
Additional Sources
In the case of multiple manuscripts that report on one study, the WWC selects one manuscript as the primary citation and lists other manuscripts that describe the study as additional sources.
-
Starkey, P., & Klein, A. (2005). A longitudinal study of the effects of a pre-kindergarten mathematics curriculum on low-income children's mathematical knowledge. Berkeley, CA: University of California.
Teaching Math to Young Children
Review Details
Reviewed: November 2013
Teaching Math to Young Children Practice Guide- Randomized Controlled Trial
- Meets WWC standards without reservations
This review may not reflect the full body of research evidence for this intervention.
Evidence Tier rating based solely on this study. This intervention may achieve a higher tier when combined with the full body of evidence.
Findings
Evidence Tier rating based solely on this study. This intervention may achieve a higher tier when combined with the full body of evidence.
Sample Characteristics
Characteristics of study sample as reported by study author.
-
Female: 48%
Male: 52% -
Urban
-
Race Asian 4% Black 53% Other or unknown 4% White 22% -
Ethnicity Hispanic 22% Not Hispanic or Latino 78%
An indicator of the effect of the intervention, the improvement index can be interpreted as the expected change in percentile rank for an average comparison group student if that student had received the intervention.
For more, please see the WWC Glossary entry for improvement index.
An outcome is the knowledge, skills, and attitudes that are attained as a result of an activity. An outcome measures is an instrument, device, or method that provides data on the outcome.
A finding that is included in the effectiveness rating. Excluded findings may include subgroups and subscales.
The sample on which the analysis was conducted.
The group to which the intervention group is compared, which may include a different intervention, business as usual, or no services.
The timing of the post-intervention outcome measure.
The number of students included in the analysis.
The mean score of students in the intervention group.
The mean score of students in the comparison group.
The WWC considers a finding to be statistically significant if the likelihood that the finding is due to chance alone, rather than a real difference, is less than five percent.
The WWC reviews studies for WWC products, Department of Education grant competitions, and IES performance measures.
The name and version of the document used to guide the review of the study.
The version of the WWC design standards used to guide the review of the study.
The result of the WWC assessment of the study. The rating is based on the strength of evidence of the effectiveness of the intervention. Studies are given a rating of Meets WWC Design Standards without Reservations, Meets WWC Design Standards with Reservations, or >Does Not Meet WWC Design Standards.
A related publication that was reviewed alongside the main study of interest.
Study findings for this report.
Based on the direction, magnitude, and statistical significance of the findings within a domain, the WWC characterizes the findings from a study as one of the following: statistically significant positive effects, substantively important positive effects, indeterminate effects, substantively important negative effects, and statistically significant negative effects. For more, please see the WWC Handbook.
The WWC may review studies for multiple purposes, including different reports and re-reviews using updated standards. Each WWC review of this study is listed in the dropdown. Details on any review may be accessed by making a selection from the drop down list.
Tier 1 Strong indicates strong evidence of effectiveness,
Tier 2 Moderate indicates moderate evidence of effectiveness, and
Tier 3 Promising indicates promising evidence of effectiveness,
as defined in the
non-regulatory guidance for ESSA
and the regulations for ED discretionary grants (EDGAR Part 77).