Evaluation of the Effectiveness of the Scholarships for Opportunity and Results Act (SOAR) Program
Contract Information
Current Status:
This study has been completed.
Duration:
June 2012 – June 2019
Cost:
$9,458,551
Contract Number:
ED-IES-12-C-0018
Contractor(s):
Westat
Pemberton Research
Contact:
The April 2011 Scholarships and Opportunities for Results (SOAR) Act provided for a five-year continuation of a school choice program for low-income residents of Washington, DC. The program, still titled the Opportunity Scholarship Program or OSP, now provides annual scholarships of about $8,000 for grades K–8 or $12,000 for grades 9–12 to enable low-income students to attend private schools in the District of Columbia in lieu of the public schools already available to them.
The previous iteration of the OSP mandated an evaluation of the program, which was completed in 2011. The SOAR Act similarly mandated an independent, rigorous evaluation of the effectiveness of the program. The law specified that the evaluation should examine the impact of being offered and of using a scholarship on key outcomes. This study is being conducted to fulfill the mandate.
- What was the impact of the OSP on student academic achievement and other measures of student success, overall and for subgroups of students identified in the statute as high priority?
- What effect did the program have on student and parent perceptions of school safety and satisfaction, and on parents' involvement in the education of their children?
The evaluation primarily compared outcomes of approximately 1,800 student applicants randomly assigned by lottery to either receive a scholarship or not receive a scholarship. Lotteries of program applicants were conducted in spring 2012 (cohort 1), spring 2013 (cohort 2), and spring 2014 (cohort 3). Data were collected for three follow-up years for each of the cohorts and for students in both the scholarship and non-scholarship groups. Data were also collected from study-administered academic assessments for math and reading, and student, parent, and principal surveys each spring (spring 2013–spring 2017).
Impact Reports
- Evaluation of the DC Opportunity Scholarship Program: Impacts Three Years After Students Applied (May 2018)
- Evaluation of the DC Opportunity Scholarship Program: Impacts Two Years After Students Applied (May 2018)
- Evaluation of the DC Opportunity Scholarship Program: Impacts After One Year (June 2017)
Descriptive Briefs and Reports
- Evaluation of the DC Opportunity Scholarship Program: Features of Schools in DC (August 2016)
- Applying to the DC Opportunity Scholarship Program: How Do Parents Rate Their Children's Current Schools at Time of Application and What Do They Want in New Schools? (April 2016)
- Evaluation of the DC Opportunity Scholarship Program: An Early Look at Applicants and Participating Schools Under the SOAR Act (October 2014)
A restricted-use file containing de-identified data is available for the purposes of replicating study findings and secondary analysis.
Key findings include:
- Three years after students applied, the OSP had no effect on student achievement. In the first two years after applying to the OSP, students offered and students using scholarships performed worse in math than those not offered scholarships. Between years two and three, growth in math scores slowed for students not offered scholarships and increased for those offered and using scholarships. As a result, the groups performed similarly by year three.
While there were no differences in achievement, three years after applying, students offered and using scholarships had lower rates of chronic absenteeism (22 and 20 percent, respectively) than did students not offered scholarships (29 percent). - After three years, the OSP improved student—but not parent—satisfaction with schools and perceptions of school safety. Students offered and students using scholarships were more likely to give their school a grade of A or B and to report that their school was "very safe" three years after they applied to the program. However, parents of students offered or using scholarships reported similar school satisfaction and perceptions of school safety as parents of students not offered scholarships.
- The OSP did not have an impact on parent involvement in education at school or at home.
Questions about the longer term effects of the OSP on reading and math achievement will remain unanswered, as Congress mandated that this evaluation conclude. However, a future evaluation will assess whether participation in the OSP affects college enrollment and completion. The next OSP evaluation will also focus on program improvement. It will examine how families experience the OSP and the supports they receive.