Educational Effects of the Tools of the Mind Curriculum: A Randomized Trial [Tools of the Mind vs. business as usual]
Barnett, W.Steven; Jung, Kwanghee; Yarosz, Donald J.; Thomas, Jessica; Hornbeck, Amy; Stechuk, Robert; Burns, Susan (2008). Early Childhood Research Quarterly, v23 n3 p299-313. Retrieved from: https://eric.ed.gov/?id=EJ807583
-
examining220Students, gradePK
Practice Guide
Review Details
Reviewed: April 2021
- Practice Guide (findings for Tools of the Mind)
- Randomized Controlled Trial
- The study does not meet WWC group design standards because it is a randomized controlled trial with high attrition, and the analytic intervention and comparison groups do not satisfy the baseline equivalence requirement.
This review may not reflect the full body of research evidence for this intervention.
Evidence Tier rating based solely on this study. This intervention may achieve a higher tier when combined with the full body of evidence.
Findings
Evidence Tier rating based solely on this study. This intervention may achieve a higher tier when combined with the full body of evidence.
Sample Characteristics
Study sample characteristics were not reported.Teaching Math to Young Children
Review Details
Reviewed: November 2013
Teaching Math to Young Children Practice Guide- Randomized Controlled Trial
- Meets WWC standards with reservations
This review may not reflect the full body of research evidence for this intervention.
Evidence Tier rating based solely on this study. This intervention may achieve a higher tier when combined with the full body of evidence.
Findings
Evidence Tier rating based solely on this study. This intervention may achieve a higher tier when combined with the full body of evidence.
Sample Characteristics
Characteristics of study sample as reported by study author.
-
Female: 47%
Male: 53% -
Urban
-
Ethnicity Hispanic 93% Not Hispanic or Latino 7%
Tools of the Mind Intervention Report - Early Childhood Education
Review Details
Reviewed: September 2008
- Randomized Controlled Trial
- Meets WWC standards without reservations
This review may not reflect the full body of research evidence for this intervention.
Evidence Tier rating based solely on this study. This intervention may achieve a higher tier when combined with the full body of evidence.
Please see the WWC summary of evidence for Tools of the Mind.
Findings
|
Outcome measure |
Comparison | Period | Sample |
Intervention mean |
Comparison mean |
Significant? |
Improvement index |
Evidence tier |
|---|---|---|---|---|---|---|---|---|
|
Wechsler Preschool Primary Scale of Intelligence (WPPSI) Animal Pegs subtest |
Tools of the Mind vs. Business as usual |
Posttest |
3-4 year olds;
|
N/A |
N/A |
No |
-- |
|
Outcome measure |
Comparison | Period | Sample |
Intervention mean |
Comparison mean |
Significant? |
Improvement index |
Evidence tier |
|---|---|---|---|---|---|---|---|---|
|
Woodcock-Johnson Revised (WJ-R): Applied Problems |
Tools of the Mind vs. Business as usual |
Posttest |
3-4 year olds;
|
N/A |
N/A |
No |
-- |
|
Outcome measure |
Comparison | Period | Sample |
Intervention mean |
Comparison mean |
Significant? |
Improvement index |
Evidence tier |
|---|---|---|---|---|---|---|---|---|
|
Peabody Picture Vocabulary Test III (PPVT-III) |
Tools of the Mind vs. Business as usual |
Posttest |
3-4 year olds;
|
N/A |
N/A |
No |
-- | |
|
Expressive One-Word Picture Vocabulary Test-Revised (EOWPVT-R) |
Tools of the Mind vs. Business as usual |
Posttest |
3-4 year olds;
|
N/A |
N/A |
No |
-- |
|
Outcome measure |
Comparison | Period | Sample |
Intervention mean |
Comparison mean |
Significant? |
Improvement index |
Evidence tier |
|---|---|---|---|---|---|---|---|---|
|
Get Ready to Read! (GRTR) |
Tools of the Mind vs. Business as usual |
Posttest |
3-4 year olds;
|
N/A |
N/A |
No |
-- | |
|
Woodcock-Johnson Revised (WJ-R): Letter-Word Identification subtest |
Tools of the Mind vs. Business as usual |
Posttest |
3-4 year olds;
|
N/A |
N/A |
No |
-- |
Evidence Tier rating based solely on this study. This intervention may achieve a higher tier when combined with the full body of evidence.
Sample Characteristics
Characteristics of study sample as reported by study author.
-
Female: 47%
Male: 53% -
Urban
-
- B
- A
- C
- D
- E
- F
- G
- I
- H
- J
- K
- L
- P
- M
- N
- O
- Q
- R
- S
- V
- U
- T
- W
- X
- Z
- Y
- a
- h
- i
- b
- d
- e
- f
- c
- g
- j
- k
- l
- m
- n
- o
- p
- q
- r
- s
- t
- u
- v
- x
- w
- y
New Jersey
-
Race Asian 4% Black 2% Other or unknown 2% -
Ethnicity Hispanic 93% Not Hispanic or Latino 7%
Study Details
Setting
This study was conducted in 18 classrooms in a low-income urban school with state-financed Abbott full-day preschool education.
Study sample
In one school selected for the study, 7 classrooms on one floor were available for Tools of the Mind implementation and 11 classrooms on another floor were available for the control condition. Teachers and assistants were randomly assigned to classrooms using a stratified assignment procedure, and then three- and four-year-old children were randomly assigned to either Tools of the Mind curriculum classrooms or district curriculum classrooms. Poverty level, achievement, and minority status were similar across intervention and comparison groups. Among the children sampled, 93 percent are Hispanic, and about 70 percent consider Spanish their primary home language. Although the overall student attrition rate was more than 25 percent, and student consent after random assignment led to differential attrition, the post-attrition intervention and comparison samples were equivalent on achievement pretests. After one year, 85 Tools of the Mind students and 117 comparison students remained in the sample.
Intervention Group
Tools of the Mind aims to aid learning and development while emphasizing emergent literacy and self-regulation. The two main goals of the curriculum are to develop underlying cognitive skills (such as self-regulation, deliberate memory, and focused attention) and to develop specific academic skills (such as symbolic thought, literacy, and an understanding of math). Play is the leading activity for developing such skills and the curriculum emphasizes the teacher’s role in supporting the development of mature intentional dramatic play. The study was conducted during the first year of program implementation of Tools of the Mind.
Comparison Group
Control classrooms implemented the standard district-created curriculum, which was described as a full-day PreK balanced literacy curriculum with themes. In structured observations of the control group, frequently observed activities were art projects that correlated with the “letter of the week,” free play, large group movement and/or music, and such large group activities as story time. According to the study authors, although the control curriculum covered much of the same academic content and topics as Tools of the Mind, there was greater emphasis on teacher-imposed control and less on children’s self-regulation.
Outcome descriptions
For both pre- and post-tests, the authors administered Peabody Picture Vocabulary Test-III, Expressive One-Word Picture Vocabulary Test-Revised, Animal Pegs Subtest of the Wechsler Preschool Primary Scale of Intelligence, and two subtests of the Woodcock-Johnson-Revised test (Applied Problems and Letter-Word Identification). Get Ready to Read! screening tool was used only at post-test assessment. IDEA Oral Language Proficiency Test was administered for the subsample of Spanish-speaking children. Problem Behaviors Scale of the Social Skills Rating System was also used in the study, but not included in this report because it was outside the scope of the Early Childhood Education review. For a more detailed description of these outcome measures, see Appendix A2.1–2.4.
Support for implementation
Teachers assigned to the Tools of the Mind group received four full days of curriculum training before the start of the school year. During the school year, they received 30-minute classroom visits approximately once a week from a Tools of the Mind trainer to address any difficulties they were having with the curriculum. In addition, Tools of the Mind teachers received 1 half-day workshop and 5 one-hour lunchtime meetings to discuss aspects of the curriculum. Control group teachers received similar amounts of training. They attended workshops on the already established district curriculum given by the district for the same amount of time.
An indicator of the effect of the intervention, the improvement index can be interpreted as the expected change in percentile rank for an average comparison group student if that student had received the intervention.
For more, please see the WWC Glossary entry for improvement index.
An outcome is the knowledge, skills, and attitudes that are attained as a result of an activity. An outcome measures is an instrument, device, or method that provides data on the outcome.
A finding that is included in the effectiveness rating. Excluded findings may include subgroups and subscales.
The sample on which the analysis was conducted.
The group to which the intervention group is compared, which may include a different intervention, business as usual, or no services.
The timing of the post-intervention outcome measure.
The number of students included in the analysis.
The mean score of students in the intervention group.
The mean score of students in the comparison group.
The WWC considers a finding to be statistically significant if the likelihood that the finding is due to chance alone, rather than a real difference, is less than five percent.
The WWC reviews studies for WWC products, Department of Education grant competitions, and IES performance measures.
The name and version of the document used to guide the review of the study.
The version of the WWC design standards used to guide the review of the study.
The result of the WWC assessment of the study. The rating is based on the strength of evidence of the effectiveness of the intervention. Studies are given a rating of Meets WWC Design Standards without Reservations, Meets WWC Design Standards with Reservations, or >Does Not Meet WWC Design Standards.
A related publication that was reviewed alongside the main study of interest.
Study findings for this report.
Based on the direction, magnitude, and statistical significance of the findings within a domain, the WWC characterizes the findings from a study as one of the following: statistically significant positive effects, substantively important positive effects, indeterminate effects, substantively important negative effects, and statistically significant negative effects. For more, please see the WWC Handbook.
The WWC may review studies for multiple purposes, including different reports and re-reviews using updated standards. Each WWC review of this study is listed in the dropdown. Details on any review may be accessed by making a selection from the drop down list.
Tier 1 Strong indicates strong evidence of effectiveness,
Tier 2 Moderate indicates moderate evidence of effectiveness, and
Tier 3 Promising indicates promising evidence of effectiveness,
as defined in the
non-regulatory guidance for ESSA
and the regulations for ED discretionary grants (EDGAR Part 77).