Co-Principal Investigators: Walters, Jonathan; Johannes, Kristen; Booth, Julie; Donovan, Suzanne

Purpose: In this project, researchers aim to test the efficacy of the MathByExample activities on improving student mathematics outcomes in 4th and 5th grade. The MathByExample intervention, developed with prior IES funding, targets key mathematics concepts and common misconceptions for 4th and 5th grade students through worked example practice exercises in which students explain a fictitious student's work (marked as either correct or incorrect) and solve a practice problem.

Project Activities: In Year 1, the research team will recruit participants and prepare the materials and protocols for the study. In Years 2 and 3, the research team will implement the study. In Year 4, the research team will finish collecting state test data and analyze the study data. In Year 5, the research team will finalize their analyses, finalize the cost analysis, and disseminate project findings.

Products: Products  will result in research findings of both practical and theoretical significance. The project team will disseminate project information and findings through reports and peer-reviewed publications to reach practitioners, researchers, and policymakers.

Structured Abstract

Setting: The study will take place in schools across the state of Maine. 57% of Maine students live in rural areas. .

Sample: During the two years of study implementation, approximately 60 school sites will participate. Within these sites, 100 4th grade teachers and 100 5th grade teachers and approximately 2,500 4th grade students and 5,000 5th grade students will participate. Approximately 42% of students in Maine are economically disadvantaged.

Intervention: The MathByExample intervention, developed with prior IES funding, targets key mathematics concepts and common misconceptions for 4th and 5th grade students through worked example practice exercises in which students explain a fictitious student's work (marked as either correct or incorrect) and solve a practice problem. MathByExample materials consist of six blocks of assignments (68 in total) for 4th grade and seven blocks of assignments for 5th grade (69 in total). Assignments are intended to span the entire school year.

Research Design and Methods: Researchers will test the efficacy of the intervention T using a three-level cluster-randomized control trial design. The research team will randomly assign half of the 60 school sites to the treatment MathByExample condition and half to the business-as-usual control condition by way of paired randomization. In the first year of implementation, 5th grade teachers will participate in the study; in the second year of implementation, 4th grade teachers will join the study in the same condition as assigned by school.

Control Condition: The control condition will be business as usual instruction.

Key Measures: The research team will collect researcher-developed measures including an assessment of procedural and conceptual knowledge, the Big Ideas Check-In assessment, an algebra readiness assessment, and a student motivation survey. The distal outcome measure is students' math scores from the end of year state standardized test in Maine, eMPowerME. The research team will also collect teacher logs, platform data, and teacher surveys to measure fidelity of implementation.

Data Analytic Strategy: To determine whether the impacts of the MathByExample treatment intervention on student learning are superior to those of the business-as-usual condition, the research team will fit three-level hierarchical linear models (HLM) to student outcomes on proximal, motivational, and distal (state test) outcome measures. Researchers will compute models separately for 4th and 5th grade samples and conduct additional analyses on the combined sample of first and second year of implementation for 5th grade students to understand the effects of teacher familiarity with the study materials on student outcomes.

Cost Analysis: The research team will conduct a cost analysis of both the intervention and the business-as-usual group. Researchers will gather costs using the "ingredients method" and will include all expenditures, such as personnel, facilities, equipment, materials, and training.

Related IES Projects: Math By Example: Dislodging Misconceptions Before They Take Root (R305A150456)