Co-Principal Investigators: Barnes, Marcia; Klein, Alice; Starkey, Prentice

Purpose: The goal of this project is to replicate the efficacy of an intensive Tier 2 math intervention, Pre-K Mathematics Tutorial (PKMT), for pre-k children at risk for mathematics difficulties, and compare the relative effectiveness of two delivery models (Pull-Out vs. In-Class) for implementing the intervention. Math difficulties occur early in childhood and are associated with negative long-term outcomes. PKMT is designed to address these difficulties by providing intensive instruction on a core set of math concepts and skills to help children at risk for mathematics disabilities build a strong foundation of mathematical knowledge. Prior research has shown that PKMT has beneficial outcomes on general mathematics outcomes for children with math difficulties when delivered as a pull-out model where researcher tutors work with small groups of students outside of the classroom. The goal of the current study is to replicate PKMT, but systematically vary the delivery model in order to compare the relative efficacy of the pull-out model versus an in-class model where pre-k teachers implement the intervention in small groups during structured time periods.

Project Activities: In this project, the research team will systematically replicate PKMT and examine the comparative efficacy of two delivery models, pull-out versus in-class in a randomized controlled trial. Assessments will be collected before and after the intervention and at 1-year follow up. Analyses will examine the impact of PKMT on children's math knowledge, compare the relative efficacy of both delivery models, and conduct exploratory analyses to examine potential moderators (attention and working memory) and mediators (dosage). The research team will also conduct an implementation study to better understand factors affecting intervention fidelity and dosage and the process for scaling a Tier 2 intervention. The cost and cost-effectiveness of PKMT relative to the control group will also be examined.

Products: The primary product of this project will be evidence of the impact of PKMT on the math outcomes of pre-kindergarten children with math difficulties and the relative efficacy of pull-out versus in-class delivery models. The project will also result in a final dataset to be shared, peer-reviewed publications and presentations, and additional dissemination products that reach education stakeholders such as practitioners and policymakers.

Structured Abstract

Setting: The study will take place in a state-funded pre-kindergarten program operated by a large urban school district in California.

Sample: Participants will include 720 children with math difficulties from 120 state-funded pre-k classrooms.

Intervention: PKMT was adapted from Pre-K Mathematics and designed to provide intensive instruction in small ability-matched groups on a core set of math concepts and skills to help children at risk for mathematics difficulties build a strong foundation of informal mathematical knowledge. It is comprised of 20 math activities that use concrete materials to promote learning of concepts related to number, arithmetic, space, geometry and measurement. PKMT incorporates a number of the key characteristics that are effective in interventions for children at-risk for mathematical disabilities, including (a) increased intensity of math instruction beyond the Tier 1 curriculum; (b) explicit, systematic instruction that integrates developmental research about mathematics with principles of direct instruction; (c) cumulative review; (d) teaching to mastery; (e) scaffolding for learning and for providing emotional support; and (f) progress monitoring to track children's understanding of each math concept and adjust instruction to their knowledge. The PKMT intervention components include curriculum materials (a scope and sequence, 20 scripted lessons with concrete manipulatives, progress monitoring tools), a training process that includes four days of professional development workshops, and fidelity supports (a fidelity of implementation form and support visits by an early intervention specialist).

Research Design and Methods: PKMT will be evaluated using a randomized controlled trial. In Year 1, the research team will prepare curriculum and assessment materials and begin recruitment. In Years 2 and 3, two cohorts of classrooms will be randomly assigned to one of three conditions, a pull-out model, an in-class model, or business-as-usual. Within these classrooms children who are identified as at risk for difficulties on a math screener will participate in the assigned condition. The team will collect baseline data on outcomes and moderators and data on outcomes after the intervention. Follow-up data on student outcomes and the types of additional instructional services that students receive will be collected a year after the intervention ends when children are in kindergarten. Fidelity data will be collected throughout intervention implementation and comparison group practice and Tier 1 math instruction will also be documented. The research team will also collect data on factors affecting fidelity and dosage, their frequency of occurrence, and tractability. In Year 4, a sub-sample of 10 teachers will engage in a continuous improvement process to facilitate the adoption and sustainability of PKMT. The research team will train district staff to conduct support visits, including fidelity and dosage checks, and provide technical assistance to teachers or other staff in screening children. In Year 5, the research team will complete all analyses, including cost and cost-effectiveness analyses, and disseminate findings.

Control Condition: Children in the control condition will receive mathematics instruction typically provided by the district.

Key Measures: Children will be screened for math risk using three tasks from the Child Math Assessment (CMA), including object counting, one-set addition and subtraction with hidden objects, and shape recognition. Child math outcomes will be assessed using the CMA and the Test of Early Mathematics Ability-3. The research team will use the Attention Sustained subtest from the Leiter International Performance Scale to assess attention and Follow Froggy to measure working memory. The Early Mathematics Classroom Observation and the accompanying Generalized Fidelity instrument will be used to document Tier 1 classroom mathematics instruction and comparison-group practice. Observational measures will be used to assess PKMT fidelity.

Data Analytic Strategy: Multilevel modeling will be used to test whether children in each of the intervention conditions outperform children in the control group and whether the delivery model for implementing PKMT has a significant impact on math outcomes. Interaction terms will be added to these models to explore the potential moderating effects of attention and working memory. Mediation analyses will be conducted to explore whether minutes of Tier 2 math instruction mediate intervention effects. Finally, the ingredients method will be used to analyze implementation costs and cost-effectiveness ratios will be computed for both delivery models of the intervention.

Related IES Projects: A Randomized Trial of a Tutor-Based Mathematics and Attention Intervention for Low-Performing Pre-Schoolers at Risk for Mathematical Difficulties in School (R324A120410)