Skip Navigation
Funding Opportunities | Search Funded Research Grants and Contracts

IES Grant

Title: A Lottery-Based Evaluation of the Impact of Public School Choice Programs on Short- and Medium-Term Academic and Behavioral Outcomes
Center: NCER Year: 2019
Principal Investigator: Betts, Julian Awardee: University of California, San Diego
Program: Improving Education Systems: Policies, Finance, Organization, Management, and Leadership      [Program Details]
Award Period: 3 years (07/01/2019 – 06/30/2022) Award Amount: $699,985
Type: Efficacy Award Number: R305A190123
Description:

Purpose: The researchers conducted a retrospective evaluation of the impact of three types of public school choice programs: magnet schools, open enrollment, and a traditional busing system designed to integrate a district's schools.

Project Activities: The research team studied school choice lottery data from a large, urban school district to examine the impact of applying to and enrolling in public schools of choice. The analysis studied the impact of winning a lottery to attend any of the three programs, as well as winning a lottery to attend each of the individual programs.  Researchers tested demographic characteristics, including race/ethnicity, English learner status, and special education needs, as moderating variables. The research team also conducted an exploratory analysis of potential school-based mediating influences, such as school-level averages of peer group characteristics, test scores and value-added in reading and mathematics, and teacher experience and other qualifications.

Key Outcomes: The main findings of this project will be shared once they are published in a publicly available peer-reviewed publication.

Structured Abstract

Setting: The project took place in the San Diego Unified School District, a large urban district in California.

Sample: The sample included K–12 students who applied to any of the three public school choice systems centrally administered by the district, over the years fall 2001 to fall 2008 and fall 2010 to fall 2013. Over this period, about 40 percent of students attended public schools of choice.

Intervention: The intervention was admission to a public choice school. During the period of study, students in the district who wished to attend a public school other than their neighborhood school had four main options available: magnet schools, the Voluntary Enrollment Exchange Program (VEEP, the traditional busing program), the School Choice program (Choice, the state-mandated open-enrollment program,), and charter schools. This study focuses on the magnet program, VEEP, and Choice.

Research Design and Methods: The team leveraged the fact that access to public choice schools in this district is lottery-based. The use of admission lotteries approximated a randomized controlled trial, which produces causal estimates. The unit of randomization is the individual student's school application. The researchers treated each school lottery across the three types of programs as a separate "experiment" and also combined the lotteries and examined the overall impact of winning a lottery and enrolling a school of choice. The researchers ensured baseline equivalence and conducted a variety of robustness checks.

Control Condition: The control condition was students in admission lotteries who did not win the lottery to attend a choice school. In addition to applying to the same school in the same year, comparison group students must also have been in the same grade level and priority group.

Key Measures: Academic outcomes included test score changes in math and English language arts (ELA), progression to the next grade, and graduation on time.  Additional measures included absences, elementary school report card ratings for perseverance and respect, secondary school "citizenship GPA", and suspensions. Postsecondary measures included enrollment in the first year after high school, the number of years enrollment in the first 4 years after high school graduation, obtaining a bachelor's degree within 5 years of high school graduation, and obtaining an associate's degree or certificate from a 2-year college within 5 years of high school graduation.

Data Analytic Strategy: Using regression techniques, the research team estimated both the impact of winning a school choice lottery (intent to treat, or ITT) and of winning a lottery and attending (treatment on the treated, or TOT). The team estimated the impact of TOT using winning a lottery as an instrumental variable for enrolling in a school receiving students as part of the given choice program. The team conducted moderation analyses by testing for interactions of winning a lottery with student background variables related to race/ethnicity and English learner and special education status.  The analyses were also repeated separately for elementary, middle and high schools. For results with robust non-zero impacts, the researchers performed a mediator analysis.  Mediating school-level variables included school demographics, school mean test scores and value-added in reading achievement tests, and teacher qualifications and demographics.

Products and Publications

Project website: https://sandera.ucsd.edu/research-projects/completed-projects/schoolchoice_overview.html

ERIC Citations:  Find available citations in ERIC for this award here.

Study Pre-registration: The study was pre-registered at the Registry of Efficacy and Effectiveness Studies. See #8140.1v1 at https://sreereg.icpsr.umich.edu/sreereg/search/search.

Publicly Available Data: The Data Analysis and Reporting Department at the San Diego Unified School District (SDUSD) has received the final dataset used for the analysis along with a data codebook. Other researchers wishing to replicate the results can do so by applying for access to the data to the RPRP (Research Proposal Review Panel) in the aforementioned office.

Additional Online Resources and Information: A discussion paper will be available at sandera.ucsd.edu.


Back