Co-Principal Investigators: Dougherty, Barbara; Harwell, Michael

Purpose: The purpose of this project is to develop and pilot test a mathematics intervention focused on whole number multiplication and division for third grade students with or at risk for mathematics difficulties (MD). Students with MD have a limited understanding of many concepts taught in early school years, especially multiplicative reasoning that is essential to solve problems in challenging situations. Although the percentage of elementary students reaching proficient levels in mathematics has increased over the past decade, as children progress into secondary grades the mathematical gains made in elementary school are often not maintained. Thus, it is critical for students with MD to develop a strong understanding of math concepts and math skills in elementary school. This project aims to address this by developing and pilot testing an intervention focused on whole number multiplication and division to improve mathematics achievement among students with MD. The project will also examine the costs of the intervention and the extent to which student characteristics, such as sex, race, and English learner status and attention moderate the intervention's effects on student math achievement.

Project Activities: The research team will develop and test the intervention over three years. In Year 1 they will develop the intervention, including professional development for teachers on how to implement the program, through interviews with district stakeholders and reviews of existing curricula. In the fall and spring researchers will conduct brief learning trials with small groups of students using the initial version of the intervention. Through these trials, researchers will test the usability of the intervention and will engage in iterative refinement based on usability data. In Year 2, researchers will revise the intervention based on data from the brief learning trials and conduct a feasibility study to test how the revised intervention functions when implemented by classroom teachers who are trained to implement the program. In Year 3, the research team will revise the intervention based on the feasibility study and conduct a pilot randomized controlled trial to evaluate its promise for improving student math achievement and collect data on the cost of implementing the intervention.

Products: The products of this project will include a fully developed mathematics intervention to improve math achievement for third graders with MD and professional development for teachers to implement the intervention. The project will result in peer-reviewed publications and presentations as well as additional dissemination products that reach education stakeholders such as practitioners and policymakers.

Structured Abstract

Setting: The research will take place in elementary schools in California.

Sample: Approximately 28 teachers who provide Tier 2 intervention and 304 third-grade students (including 120-140 students with MD) will participate in this research over the 3-year period. Students scoring below the 25th percentile on the Concepts and Applications subtest of the AimswebPlus will be considered to have MD for purposes of this study.

Intervention: The intervention is a supplemental instructional program that focuses on (a) the meaning of multiplication; (b) multiplicative relations; and (c) whole number concepts, operations, and properties within word problem contexts. The intervention includes mathematics content integrated with Common Core State Standards for Mathematical Practice, instruction centered on problem solving, and use of evidence-based instructional practices, such as explicit instruction, visual representations of concepts, and practice activities with feedback. Implementation of the intervention is carried out by classroom teachers 5 days a week for 30 minutes per day. Teachers receive 3 days of in-person professional development and ongoing coaching and have access to a Teacher Guide. The goals of the PD include the following: (a) to model instructional techniques for multiplicative reasoning; (b) to investigate students' misconceptions about multiplicative relationships; (c) to orient teachers to the intervention and how to navigate the Teacher Guide to implement the activities and tasks; (d) to provide guidance for implementing instructional practices that are integral to the intervention, such as explicit instruction and scaffolding; and (e) to create questions that expose student understanding and types of scaffolding questions and representations to improve student performance.

Research Design and Methods: The researchers will use an iterative research and development process over the course of three years to develop and test the mathematics intervention. In Year 1, the research team will develop the intervention and professional development program by conducting interviews with district stakeholders to understand areas of student need and reviewing existing curricula to create a set of target student outcomes and a scope and sequence of lessons and activities. Researchers will also conduct a brief learning trial with a small group of teachers and students with MD to test the usability of the intervention. Fidelity will be assessed through observational and audio-recorded data and teachers will report on the usability of the intervention through focus groups. This data will inform ongoing refinement of the intervention. In Year 2, researchers will conduct a feasibility study with teachers and students with or at risk for MD. Researchers will provide professional development, coaching, and support teachers implementing the intervention and collect data on fidelity and feasibility using the same methods as Year 1. In Year 3 the research team will conduct a pilot study using a small randomized control trial with students nested in classes to assess fidelity of implementation and the promise of the intervention for improving math outcomes for third-grade students receiving Tier 2 instruction (approximately one-third of whom are anticipated to be at risk for MD). Analyses will explore the effects of the intervention, in addition to the intervention's cost, and the extent to which student characteristics, such as sex, race, and English learner status and attention moderate the intervention's effects on student math achievement.

Control Condition: In the pilot study, teachers in the control condition will carry out business-as-usual math instruction.

Key Measures: Student outcome measures include a researcher-developed measure of multiplicative reasoning, the Process and Computation subtest from the Group Mathematics Assessment and Diagnostic Evaluation (GMADE), and the end-of-year Smarter Balanced Assessment to assess of mathematics achievement. The researcher-developed measure and the GMADE will be administered by classroom teachers in a whole-classroom format. Attention will be measured with the Strengths and Weaknesses of Attention-Deficit/Hyperactivity-symptoms and Normal-behaviors rating scale. Researchers will also use teacher surveys to understand feasibility of implementation and student engagement, student surveys to understand intervention acceptability, focus groups and interviews to understand usability and feasibility of the intervention, and a tool to score audio recordings and direct observation data to measure fidelity of implementation. They will also collect data from participating districts, schools, and teachers regarding the costs associated with implementing the intervention, such as expenditures for personnel, facilities, equipment, materials, and professional development.

Data Analytic Strategy: Researchers will descriptively analyze survey data and code the qualitative data for themes to inform the development of the intervention and to understand its feasibility and usability. Multilevel models with students nested in classrooms will be used to analyze data from the Year 3 pilot study. Researchers will also conduct moderation analyses to explore whether the effects of the intervention are influenced by student characteristics and attention. Researchers will calculate costs for implementing the intervention in one school based on data collected from all participating districts and schools.