Purpose: The researchers will conduct a retrospective evaluation of the impact of three types of public school choice programs: magnet schools, open enrollment, and a traditional busing system designed to integrate a district's schools.

Project Activities: The research team will study almost two decades of school choice lottery data from a large, urban school district to examine the impact of applying to and enrolling in public schools of choice. They will examine the effect of school characteristics such as teacher qualifications, school climate, and peer groups at different types of choice schools on the educational experiences and outcomes of students. Researchers will test demographic characteristics, including parental education and race/ethnicity, English Learner status, and special education needs, as moderating variables. The research team will also conduct an exploratory analysis of potential school-based mediating influences, such as school means of peer group characteristics, school value-added in reading and mathematics, student perceptions of school climate, and teacher experience and qualifications.

Products: Researchers will produce evidence of the differential impact of, as well as mediators and moderators of, public school choice on student outcomes. The team will also produce a cost effectiveness analysis, peer-reviewed publications and a final publicly accessible dataset.

Structured Abstract

Setting: The project takes place in a large urban district in California.

Sample: The sample for the main analysis includes K-12 students who applied to any of the three public school choice systems centrally administered by the district, over two decades: 2001 to 2021. At the midpoint (2009-10), about 40% of students attended public schools of choice.

Intervention: The intervention for this study is admission to a public choice school. During the period of study, students in the district who wished to attend a public school other than their neighborhood school had four main options available: magnet schools, the Voluntary Enrollment Exchange Program (VEEP), the School Choice program (the state-mandated open-enrollment program), and charter schools. In addition, Program Improvement School Choice (PISC), initiated in fall 2002, responded to the No Child Left Behind (NCLB) requirements to provide choice options to students at schools that failed to make Adequate Yearly Progress (AYP) in a given subject for two consecutive years by using or adapting the district's existing busing routes under the VEEP and magnet programs to afford school choice to students who qualified under NCLB. Students who were eligible for PISC were given a higher priority group within the VEEP and magnet programs and were more likely to gain admission than other students.

Research Design and Methods: In this study, the team will leverage the fact that access to public choice schools in this district are lottery-based. The use of admission lotteries approximates a randomized controlled trial, which produces causal estimates. The unit of randomization is the individual student application. The researchers will treat each school lottery across the three types of programs as a separate “experiment” but will also combine the lotteries and examine the overall impact of winning a lottery and enrolling a school of choice. In addition to applying to the same school in the same year, students must also be in the same grade level and priority group to be compared against one another. The researchers will examine baseline equivalence and conduct a variety of robustness checks.

Control Condition: The control condition is students in admission lotteries who do not win the lottery to attend a choice school.

Key Measures: Short-term outcomes include test score changes and progression to the next grade. Longer term outcomes include completion of the expected number of courses with a high enough GPA in grade 9 to be deemed on track to graduate, passing the state's exit examination, graduating on time, and measures of postsecondary enrollment and postsecondary degrees obtained. Non-cognitive outcomes include the percentage of days absent, suspensions, citizenship grades at the secondary school level, and behavioral measures from elementary school report cards. Mediating school level variables include three measures of school climate derived from the California Healthy Kids Survey, school demographics, school value-added in reading and mathematics achievement tests, high school course offerings and mean teacher experience and qualifications.

Data Analytic Strategy: Using regression techniques, the research team will estimate both the impact of winning a school choice lottery (intent to treat, or ITT) and of winning a lottery and attending (treatment on the treated, or TOT). They will estimate the impact of TOT using winning a lottery as an instrumental variable (IV) for enrolling in a school receiving students as part of the given choice program. They will conduct moderating analyses by testing for interactions of winning a lottery with student background variables.