August 2021 – August 2026
The coronavirus pandemic led to substantial unfinished learning in math and an important debate about how best to address it. Traditionally, policymakers and educators have advocated a "mastery" approach, which views all below-grade content as foundational and systematically reteaches any content where there are knowledge gaps. "Just-in-time" learning has received attention more recently, including in the U.S. Department of Education's COVID-19 Handbook. This alternative reteaches only below-grade content deemed most essential to understanding the current grade-level topic. But there is limited evidence on which approach is most effective for which students and which contexts. This evaluation will examine the effectiveness of adaptive technology products that deliver these two catch-up strategies in elementary schools, where teachers often struggle with how to teach math well and the benefits of using technology supports are understudied. The findings will provide valuable evidence, especially for low-performing schools identified under the Every Student Succeeds Act and their most underserved students.
This impact evaluation will assess the effectiveness of at least two adaptive math technology products, that provide a "just-in-time" and a "mastery" catch-up learning approach. Up to 150 schools nationally will be recruited in school year 2022–23 to participate in the study. Schools will select which of the technology products they wish to implement. Schools will be assigned by lottery to either implement their chosen product in fourth and fifth grade for two school years (2023–24 and 2024–25) or continue with typical practice. In each school that is assigned to implement the technology product, students will be assigned by lottery to implement the technology product using either the "just-in-time" or "mastery" approach.
Data collection will include: a teacher survey to examine the effects of the technology on classroom practice; data from the technology platform to examine the effects on student engagement and implementation; a study-administered math assessment; and district administrative data on students' math and reading achievement.
The first report for the study is expected in 2024 and will be announced on http://ies.ed.gov/ncee/.
Key findings will be available after the study report is published.