Project Activities
The project will include 60 schools and 240 teachers serving 6,000 students in Grades 4 and 5. Researchers will use a cluster, multisite field trial that randomly assigns schools with equal probability to treatment—participation in SIM PD—or a delayed-treatment control condition. The study will collect classroom observations and end-of-year student performance on statewide math assessments to assess the impacts of SIM PD. The study also will examine implementation, service contrast, and costs, using information collected from PD observations, coaching logs, teacher surveys, district and school administrator interviews, and extant cost and pricing records. Last, the study will explore student perspectives through student focus groups.
Structured Abstract
Setting
The project will focus on districts that primarily serve underserved students. The first cohort of schools will be in a mid-sized urban district in New Mexico serving predominantly Latinx students and a large suburban district in Georgia serving predominantly African American students.
Sample
Teachers involved in the study will provide primary math instruction in Grades 4 and 5, serving students in general grade-level classes.
SIM PD uses workshops, practice sessions, and group debriefing meetings to support teachers in using research-based questioning and discourse facilitation strategies. The activities are organized into three modules, each focused on a different instructional strategy. Within each module, teachers from the same school participate in workshops to learn about a focal strategy, and the research that supports it, and then prepare to practice it. During practice sessions in the mixed-reality classroom, teachers try the strategy with student avatars, pause to get feedback and reflect on progress with a coach and other teachers, identify next steps, and try again. Teachers practice in the mixed-reality classroom first with their school team and then individually. Last, teachers meet again to reflect on their experiences and discuss how they have and will continue to transfer their learning to their classrooms.
Research design and methods
Researchers will use a multisite, cluster randomized trial designed to meet What Works Clearinghouse standards without reservations. The trial will involve 60 schools and about 240 teachers across two cohorts. Schools within each cohort will be randomly assigned with equal probability to the treatment group or the delayed-treatment control group within each study district at the beginning (early fall) of the intervention year. Researchers also will conduct an implementation study using PD artifacts and observations, administrator interviews, and teacher surveys, and explore student experiences through student focus groups.
Control condition
Participating teachers at schools in the control condition will continue with the business-as-usual PD offering available through their schools and districts. In the year after evaluation activities, these teachers will be offered participation in SIM PD.
Key measures
To assess program impact on the quality of classroom math discussion, the evaluation team will collect video classroom observations at three time points for each teacher in the study: a baseline observation in the fall of the intervention year before SIM PD begins (fall 2023 for Cohort 1 and fall 2024 for Cohort 2), and two observations immediately after the PD ends in the spring (spring 2024 or 2025). Videos will be coded remotely by math instruction experts blind to each teacher's treatment condition, using scoring codes adapted from the Mathematics Scan observation rubric that focus on features of mathematics group discussion.
To examine student math achievement for both cohorts, the evaluation team will collect fourth- and fifth-grade state math assessment scores from spring 2024 (Cohort 1) and spring 2025 (Cohort 2). Assessment scores across districts will be standardized using grade-level means and standard deviations of the test scores within each state.
Data analytic strategy
Impacts on the quality of classroom math discussion and on student achievement will be estimated using multilevel modeling. Researchers will explore potential moderation effects of key student and teacher characteristics, and examine whether the impact of SIM PD on student math achievement is mediated by the quality of classroom math discussion. To help interpret impact findings, researchers will assess implementation fidelity and explore how implementation fidelity is associated with teacher and student outcomes based on data from the treatment group.
Cost analysis strategy
Researchers will conduct cost analyses using a resource cost model that uses an "ingredients" approach to capture systematically all personnel and non-personnel costs for implementing an intervention. The cost-analysis results will be combined with the impact findings to conduct a cost-effectiveness analysis that captures the cost-per-standard-deviation impact of the PD. Cost-effectiveness will be calculated both per-teacher and per-student.
People and institutions involved
IES program contact(s)
Project contributors
Products and publications
Products: Researchers plan to disseminate the findings and implications for practice to a diverse audience through journal publications, research briefs, webinars, practitioner papers, conference presentations, and postings to social media.
Questions about this project?
To answer additional questions about this project or provide feedback, please contact the program officer.
