Project Activities
The researchers worked with the Los Angeles Unified School District (LAUSD) to identify, recruit and confirm eligible schools for study participation. During the summers of 2018 and 2019, researchers worked with the district and study schools to identify and randomly assign students who had completed their first year of high school but who failed at least one of two core courses in ninth grade to one of two conditions: online credit recovery (treatment) or repeating the standard teacher-directed course (control). The researchers collected and analyzed both the primary data sources (an end-of-course assessment, a teacher survey, a monthly teacher log, and a student survey) and extant data sources (PSAT data; enrollment and course data; and graduation data) to determine the effect of online credit recovery courses and how instruction in these courses compares with teacher-directed credit recovery courses. They also supplemented the primary project activities with teacher focus groups, student interviews, and a national survey of district administrators.
Structured Abstract
Setting
The study took place in 24 high schools in the Los Angeles Unified School District (LAUSD) in California.
Sample
The study focused on 1,683 students who completed their first year of high school in the 2017-18 or 2018-19 school year and enrolled in an Algebra 1 or 9th grade English (English 9) credit recovery course during the summer 2018 or summer 2019 term.
The study tested an online learning model for Algebra 1 and English 9 credit recovery classes, where an online provider supplied the main course content and the school provided a subject-appropriate, credentialed in-class teacher who could supplement the digital content with additional instruction. The online classes took place in standard high school classrooms.
Research design and methods
The researchers used a block randomized design with student-level random assignment within schools, where outcomes for students assigned to online credit recovery were compared to outcomes for students assigned to typical teacher-directed credit recovery. The researchers used both primary data collection and extant data to analyze measures of student content knowledge, student credit accumulation, high school graduation, implementation fidelity in the treatment classrooms, instructional features of the treatment and control classrooms, and student instructional experiences. They also conducted a cost analysis.
Control condition
The control condition was defined as the typical (business-as-usual) teacher-directed credit recovery classes students take at each high school in the study, which included a subject-appropriate, credentialed in-class teacher in standard high school classrooms.
Key measures
The key outcome measures were student content knowledge, measured with a study-developed end-of-course assessment and grade 10 PSAT scores, final grade in the credit recovery course, student credit accumulation over four years of high school, and on-time high school graduation, measured with district administrative records. The researchers also used a student survey to measure students' experiences in their credit recovery course and a teacher survey to measure implementation.
Data analytic strategy
The primary analyses focused on intent-to-treat effects on student instructional experiences, test scores, credit accumulation, and high school graduation. The research team conducted the main analyses separately for Algebra 1 and English 9 courses. An additional set of analyses compared the key features of the online course to the features of the teacher-directed course, including the costs of implementing each type of course.
Cost analysis strategy
Using the ingredients method, the researchers found that differences in the comprehensive costs for online and teacher-directed credit recovery classes are small. However, cost differences across the two models of specific types of resources are substantial and statistically significant. Online credit recovery classes resulted in higher district-incurred costs largely due to the purchase of online curricula and required technology. The online model incurred lower opportunity costs to teachers stemming from less out-of-class time devoted to grading, lesson planning, and developing course materials (Atchison et al., 2024).
Key outcomes
- For both Algebra 1 and English 9, having an online credit recovery class instead of a teacher-directed class did not significantly affect students’ content knowledge (Rickles et al., 2024).
- The credit recovery rate in English 9 online classes was 15 percentage points lower than in teacher-directed classes. For Algebra 1, the credit recovery rate was lower in the online classes, but not significantly so (7 percentage points lower) (Rickles et al., 2024).
- Compared to teacher-directed credit recovery classes, online credit recovery classes resulted in higher district-incurred costs largely due to the purchase of online curricula and required technology. However, the online classes had lower opportunity costs to teachers stemming from less out-of-class time devoted to grading, lesson planning, and developing course materials (Atchison et al., 2024).
People and institutions involved
IES program contact(s)
Products and publications
Products: The products of this project include evidence about the impact of online credit recovery courses in Algebra I and English 9 on student education outcomes; a description of online and f2f instruction; a cost analysis; a de-identified, restricted-use final dataset with documentation; and peer reviewed publications.
Project website:
Study registration:
Publications:
ERIC Citations: Find available citations in ERIC for this award here.
Select Publications:
Rickles, J., Clements, M., Brodziak de los Reyes, I., Lachowicz, M., Lin, S., & Heppen, J. (2024). A multisite randomized study of an online learning approach to high school credit recovery: Effects on student experiences and proximal outcomes. Journal of Research on Educational Effectiveness, 17(3), 467-490.
Atchison, D., Clements, M., Rickles, J., Brodziak de los Reyes, I., & Heppen, J. (2024). Comparing the costs of online and teacher-directed credit recovery. Educational Policy, 38(6), 1487-1514.
Available data:
A public-use deidentified data file for the impact study is available through the Inter-university Consortium for Political and Social Research (ICPSR) data repository.
Supplemental information
Co-Principal Investigator: Cynthia Lim (Los Angeles Unified School District)
Questions about this project?
To answer additional questions about this project or provide feedback, please contact the program officer.
