Co-Principal Investigator: Cynthia Lim (Los Angeles Unified School District)

Purpose: In this project, researchers will investigate how online credit recovery affects student content knowledge, credit accumulation, and graduation, relative to retaking failed courses in a standard face-to-face format. They will also describe the ways in which online credit recovery provides students with different instructional experiences compared to repeating the standard face-to-face (f2f) course. School districts across the country are increasingly using online courses to expand credit recovery options for high school students who need to get back on track toward graduation. However, the growing use of online credit recovery for high school students has outpaced the research. As concerns mount over whether students actually learn in online courses, and as questions arise about how to best implement online credit recovery, there is a critical need for rigorous evidence about the effective use of online credit recovery for high school students. This study is designed to address these questions in a large, urban school district, focusing on two common credit recovery courses: Algebra I and English 9.

Project Activities: The researchers will work with LAUSD to identify and recruit eligible schools and recruit/confirm schools for study participation. During the summer after each of two cohort's first year of high school (2017 and 2018), the study team will work with the district and study schools to identify and randomly assign students who failed at least one of two core courses in ninth grade to one of two conditions: online credit recovery (treatment) or repeating the standard f2f course (control). Researchers will collect and analyze both the primary data sources (an end-of-course assessment, a teacher survey, a monthly teacher log, and a student survey) and extant data sources (state assessment data; district student demographic, enrollment, and course data; and student course progress data) to determine the effect of online credit recovery courses and how instruction in these courses compares with f2f credit recovery courses.

Products: The products of this project include evidence about the impact of online credit recovery courses in Algebra I and English 9 on student education outcomes; a description of online and f2f instruction; a cost analysis; a de-identified, restricted-use final dataset with documentation; and peer reviewed publications.

Structured Abstract

Setting: The study will take place in 15 high schools in the Los Angeles Unified School District (LAUSD) in California.

Sample: The study will focus on approximately 3,000 students who enter 10th grade in the 2017–18 or 2018–19 school year and need to recover Algebra I and/or English 9 course credit.

Intervention: The intervention is a blended-learning model of online credit recovery, including in-class instructional support, opportunities for remediation, and interactive content that offers immediate feedback. The treatment condition is defined as being assigned to recover Algebra I and/or English 9 credit using the blended model, during the regular school year, in grade 10.

Control Condition: The control condition is defined as being assigned to recover Algebra I and/or English 9 credit in typical, business-as-usual courses delivered in a face-to-face format during the regular school year, in grade 10.

Research Design and Methods: The study will use a block randomized design with student-level random assignment within schools, where outcomes for students assigned to online credit recovery will be compared to outcomes for students assigned to standard, face-to-face credit recovery. The study will use both primary data collection and extant data to analyze measures of student content knowledge, student credit accumulation, high school graduation, implementation fidelity in the treatment classrooms, instructional features of the treatment and control classrooms, and student instructional experiences. The project also includes a cost analysis.

Key Measures: The key outcomes are student content knowledge, measured with an end-of-course assessment and the state-required grade 11 Smarter Balanced math and English/language arts assessments, and student credit accumulation and on-time graduation, measured with district administrative records. The study will also use a student survey to measure students' experiences in their credit recovery course.

Data Analytic Strategy: The primary analyses will focus on intent-to-treat effects on student instructional experiences, test scores, credit accumulation, and high school graduation. The research team will conduct the main analyses separately for Algebra I and English 9 courses, with supplemental analyses based on the pooled sample. An additional set of analyses will examine the extent to which the online credit recovery course was implemented as intended and will compare the key features of the online course to the features of the face-to-face course.

Project Website: https://www.air.org/project/assessing-efficacy-online-credit-recovery-student-learning-and-high-school-graduation