Project Activities
The research team developed and tested the intervention over 3 years. In year 1 they developed the intervention, including professional development for teachers on how to implement the program, through interviews with district stakeholders and reviews of existing curricula. Researchers conducted brief learning trials with small groups of students using the initial version of the intervention. Through these trials, researchers tested the usability of the intervention and engaged in iterative refinement based on usability data. In year 2, researchers revised the intervention based on data from the brief learning trials and conducted a feasibility study to test how the revised intervention functions when implemented by classroom teachers who are trained to implement the program. In year 3, the research team revised the intervention based on the feasibility study and conducted a pilot randomized controlled trial to evaluate its promise for improving student math achievement and collected data on the cost of implementing the intervention.
Structured Abstract
Setting
The research took place in elementary schools in California.
Sample
There were 22 teachers who provide Tier 2 intervention and 127 third grade students with MD who participated in this research over the 3-year period. For the purpose of this study, we identified students experiencing MD as those students likely in need of strategic support based on their scores on the Acadience® Math Concepts & Applications Benchmark assessment.
The intervention is a supplemental instructional program that focuses on (a) the meaning of multiplication; (b) concepts of multiplication; and (c) concepts of division and patterns. The intervention includes mathematics content integrated with Common Core State Standards for Mathematical Practice, instruction centered on problem solving, and use of evidence-based instructional practices, such as explicit instruction, visual representations of concepts, and practice activities with feedback. Implementation of the intervention is carried out by classroom teachers 5 days a week for 30 minutes per day. Teachers receive 3 days of in-person professional development and ongoing coaching and have access to a Teacher Guide. The goals of the PD include the following: (a) to model instructional techniques for multiplicative reasoning; (b) to investigate students' misconceptions about multiplicative relationships; (c) to orient teachers to the intervention and how to navigate the Teacher Guide to implement the activities and tasks; (d) to provide guidance for implementing instructional practices that are integral to the intervention, such as explicit instruction and scaffolding; and (e) to create questions that expose student understanding and types of scaffolding questions and representations to improve student performance.
Research design and methods
The researchers used an iterative research and development process over the course of 3 years to develop and test the mathematics intervention. In the first year, the research team developed the intervention and professional development program by conducting interviews with district stakeholders to understand areas of student need and reviewed existing curricula to create a set of target student outcomes and a scope and sequence of lessons and activities. Researchers conducted brief learning trials with a small group of teachers and students with MD to test the usability of the intervention. Fidelity was assessed through observational and audio-recorded data and teachers reported on the usability of the intervention through focus groups. This data informed ongoing refinement of the intervention. The following year, researchers conducted a feasibility study with teachers and students with or at risk for MD. Researchers provided professional development, coaching, and support to teachers implementing the intervention and collected data on fidelity and feasibility using the same methods as Year 1. Finally, the research team conducted a pilot study using a small, randomized control trial with students nested in classes to assess fidelity of implementation and the promise of the intervention for improving math outcomes for third-grade students receiving Tier 2 instruction. Analyses explored the effects of the intervention, the intervention's cost, and the extent to which student characteristics–such as sex, race, and English learner status–and attention moderated the intervention's effects on student math achievement.
Control condition
In the pilot study, teachers in the control condition caried out business-as-usual math instruction.
Key measures
Student outcome measures included a researcher-developed measure of multiplicative reasoning, the Process and Computation subtest from the Group Mathematics Assessment and Diagnostic Evaluation (GMADE), and the end-of-year Smarter Balanced Assessment to assess mathematics achievement. The researcher-developed measure and the GMADE were administered by classroom teachers in a whole-classroom format. Attention was measured with the Strengths and Weaknesses of Attention-Deficit/Hyperactivity-symptoms and Normal-behaviors (SWAN) rating scale. Researchers used teacher surveys to understand feasibility of implementation and student engagement, student surveys to understand intervention acceptability, focus groups and interviews to understand usability and feasibility of the intervention, and a tool to score audio recordings and direct observation data to measure fidelity of implementation. They also collected data from participating districts, schools, and teachers regarding the costs associated with implementing the intervention, such as expenditures for personnel, facilities, equipment, materials, and professional development.
Data analytic strategy
Researchers descriptively analyzed survey data and coded the qualitative data for themes to inform the development of the intervention and to understand its feasibility and usability. Multilevel models with students nested in classrooms were used to analyze data from the pilot study. Researchers also conducted moderation analyses to explore whether the effects of the intervention were influenced by student characteristics and attention.
Cost analysis strategy
Researchers calculated costs for implementing the intervention for one teacher in one school based on data collected from all participating districts and schools. Overall, the cost for one teacher to implement the intervention with 6 students is approximately $6,300.
Key outcomes
The main findings of this project, as reported by the principal investigator, are as follows:
- Classes that participated in the multiplicative reasoning intervention scored significantly higher than control classrooms on the end-of-year multiplicative reasoning posttest scores. However, this was not the case for the GMADE assessment or the SWAN outcome measure.
- Native English speakers were found to be more likely to have more inattentive behaviors than English Learners.
- The results suggest that students’ performance improved with regard to understanding the underlying concepts of whole number multiplication and division and applying appropriate strategies to solve multiplication and division problems.
- Both students and teachers responded positively with regards to the acceptability of the multiplicative reasoning intervention.
People and institutions involved
IES program contact(s)
Project contributors
Products and publications
Project website:
Publications:
ERIC Citations: Find available citations in ERIC for this award here.
Select Publications:
Jitendra, A. K., Dougherty, B., Sanchez, V., Harwell, M., & Harbour, S. (2023). Building conceptual understanding of multiplicative reasoning content in third graders struggling to learn mathematics. Learning Disabilities Research & Practice. Advanced Online Publication. https://doi.org/10.1111/ldrp.12322
Jitendra, A. K., Dougherty, B., Sanchez, V., & Suchilt, L. (2022). Using multiple representations to foster multiplicative reasoning in students with mathematics learning disabilities. TEACHING Exceptional Children. Advanced Online Publication. https://doi.org/10.1177/00400599221115627
Questions about this project?
To answer additional questions about this project or provide feedback, please contact the program officer.
