Co-Principal Investigator: Susanna Loeb (Stanford University)

Purpose: This project explores how taking virtual (online) courses in high school affects students' course progression and academic achievement. Online courses are widely used, but policymakers have little rigorous evidence on whether online courses promote student achievement. Some states, such as Florida, require all students to take at least one online course prior to graduation. This project will use data from Florida to examine the use of online courses, potential effects of online courses on student achievement, and potential reasons for those effects. Specifically, the study will answer the following research questions:

• Which students are most likely to take online courses? Which types of courses are students most likely to take online?
• How are access to online classes and taking of online classes associated with the course progression and academic achievement of students, and how does that differ across student groups?
• What mechanisms explain why some students enroll in online courses and some do not, and what mechanisms explain potentially different effects of online course-taking across students?

Project Activities: Researchers will use statistical modeling to analyze student data from the Florida Virtual School (FLVS), which is the largest online education provider in Florida, Miami-Dade Public Schools (MDPS), and the Florida Department of Education (FDOE) to compare outcomes for high school students taking courses in the same subjects online versus face-to-face (FtF). The researchers will also conduct surveys with students and teachers in virtual and FtF schools to supplement the main analyses and aid in interpreting the findings.

Products: The products of this project will be preliminary evidence of how virtual course-taking is related to student outcomes and peer reviewed publications.

Structured Abstract

Setting: This project will take place in Florida, and data from three sources will be linked for analysis: FLVS, MDPS, and FDOE. Researchers will also collect surveys from students and teachers in FLVS and MDPS.

Sample: Secondary analyses will rely on administrative data from all middle- and high-school students and teachers in Florida public schools (both virtual and FtF). This will comprise over 15.3 million student-years and 850,000 teacher-years of observation. Surveys will also be collected from 800 teachers and 3000 students in in FLVS and MDPS.

Intervention: The study will examine virtual and FtF course taking. The virtual courses are offered through FLVS, a public school that provides online, self-paced courses that students can access from home, a computer lab, or school library.

Research Design and Methods: The research plan includes primary data collection and analysis and secondary data analysis. Researchers will first use a series of statistical models to describe who takes virtual courses and the differences in student outcomes associated with virtual course access and course-taking. Researchers will also explore potential mediating mechanisms of the effects, such as teacher quality, peer quality, and curricular/instructional approaches. Outcomes include student test outcomes, course grades, and course progression. The teacher and student surveys will offer insights into motivations and processes that will help the researchers to generate hypotheses about the causes of the observed patterns.

Control Condition: For the secondary data analysis, students in FLVS schools will be compared with students taking courses in the same subjects in FtF schools. For the survey, researchers will use matched comparison groups of students from FDOE and FLVS schools, and from FLVS and MDPS schools, who are taking the same courses in different formats.

Key Measures: The secondary data analysis will rely on administrative data that has already been collected, including student scores on Florida's state standardized test, the FCAT, FCAT 2.0, and End-of-Course tests. Value-added measures will be constructed for each teacher. The researchers will develop the surveys based on well-validated questions from prior surveys and conduct pilot tests to refine the surveys.

Data Analytic Strategy: Researchers will use multiple analytic strategies, including descriptive techniques (e.g., crosstabs), multiple regression, student fixed effects, difference-in-difference, and value-added models.

Products and Publications

Journal article, monograph, or newsletter

Jacob, B, Berger, D, Hart, C, and Loeb, S. (2016). Can Technology Help Promote Equality of Educational Opportunities?. RSF: The Russell Sage Foundation Journal of the Social Sciences, 2 (5): 242–271.