Project Activities
Investigators will iteratively co-design and pilot test adaptive supports for struggling readers embedded with Carnegie Learning's MATHia student software and for teachers within the LiveLab teacher app for real-time classroom monitoring. In Years 1–2 of the project, investigators will work with teachers and middle school students from three suburban and urban public school districts. The research team will conduct a pilot study in Year 3 using a cluster randomized controlled trial with teachers and middle school students from five suburban and urban districts. Teachers will be randomly assigned to either use MATHia and LiveLab with MARACAS supports or without these developed supports, blocked by school-district and school. The final sixth months of the project are devoted to analyzing data from the pilot study and dissemination activities.
Structured Abstract
Setting
The research will take place in middle schools in six public school districts across urban and suburban settings in California, Florida, Massachusetts, and Pennsylvania.
Sample
During Years 1-2, the research team will work with middle school students (grades 6-8) with reading difficulties (RD) and their math teachers in inclusive classrooms. The pilot study in Year 3 will include approximately 10,000 students from 79 middle schools across five school districts, largely representative of diverse urban and suburban districts across the country. Students with reading difficulties will be defined as students who score below the 25th percentile on their state's English Language Arts Exam in the 2022-23 school year.
Phase 1 will develop a fully functional MARACAS prototype, a version of MATHia with five real-time supports for students: highlighting conceptual relationships, contextualizing/decontextualizing problems, adjusting text readability, advanced read aloud and synchronized highlighting, and personalized interest areas and names. The project will also include the iterative development of a version of LiveLab with alerts for teachers and a professional learning program.
Research design and methods
In Phase 1 (Year 1–2), MARACAS will be developed through an iterative, design-based implementation research process with students and teachers. Minimally viable components will be rapidly prototyped and evaluated through continuous cycles of design, implementation, analysis, and redesign. Observations, surveys, focus groups, informal interviews, usage log data, and cognitive labs will be used to evaluate the clarity, accessibility, use, value, and promise of MARACAS. During the Phase 2 (Year 3) pilot study, a cluster randomized control trial, with assignment at the teacher level (blocked by district and school), will be used to determine the extent to which MARACAS increases mathematics learning gains of students with RD compared to the control condition and whether such gains are mediated by MATHia knowledge components that are related to reading comprehension.
Control condition
Students with RD will use a "business-as-usual" version of MATHia, without MARACAS components. Their teachers will use a version of LiveLab without MARACAS components and will receive professional learning without additional MARACAS Materials.
Key measures
Student outcome measures include each state's standardized end-of-year exams for mathematics, students' mathematics course pass rates, and the rate at which students complete MATHia mathematics content. Data collected from observations, surveys, focus groups, informal interviews, cognitive labs, and usage data from MATHia and LiveLab will be used to understand use, usability, acceptance, the promise of supports, and any areas where those supports can be improved.
Data analytic strategy
Researchers will use multilevel modeling to test for the main treatment effect of MARACAS and moderator effects of various teacher and student characteristics controlling for covariates. The research team will also analyze individual- and group-level mediation effects. They will conduct an instrumental variable analysis to evaluate the relationship between implementation fidelity and student outcomes.
Cost analysis strategy
The proposed project includes a plan for a cost analysis based on data collected during the Phase 2 pilot using the ingredients method to understand the costs associated with implementing MARACAS beyond what schools typically spend. Cost analysis will consider both national and local costs as well as total, average, and incremental costs. Researchers will also explore the cost-effectiveness of MATHia and LiveLab with MARACAS as compared to MATHia and LiveLab implemented without MARACAS using cost-effectiveness ratios.
People and institutions involved
IES program contact(s)
Project contributors
Products and publications
Products: This project will result in fully developed MARACAS reading support features embedded in Carnegie Learning's commercially available adaptive mathematics student software MATHia as well as the LiveLab teacher application and related professional development materials to decrease reading challenges. The project will result in peer-reviewed publications and presentations as well as additional dissemination products that reach education stakeholders, such as practitioners and policymakers. To support use of project findings, generated technical resources, such as design assets and heuristics, will be Creative Commons licensed and made available through GitHub.
ERIC Citations: Find available citations in ERIC for this award here.
Related projects
Questions about this project?
To answer additional questions about this project or provide feedback, please contact the program officer.
