Executive Summary
The District of Columbia School Choice Incentive Act of 2003, passed by
Congress in January 2004, established the first federally funded, private school
voucher program in the United States. As part of this legislation, Congress mandated
a rigorous evaluation of the impacts of the Program, now called the DC Opportunity
Scholarship Program (OSP). This report presents findings from the evaluation of
the impacts 3 years after families who applied were given the option to move from
a public school to a participating private school of their choice.
The evaluation is based on a randomized controlled trial design that compares the
outcomes of eligible applicants randomly assigned to receive (treatment group) or
not receive (control group) a scholarship through a series of lotteries. The main
findings of the evaluation so far include:
- After 3 years, there was a statistically significant positive impact on
reading test scores, but not math test scores. Overall, those offered a
scholarship were performing at statistically higher levels in reading—equivalent
to 3.1 months of additional learning—but at similar levels in math compared
to students not offered a scholarship (table 3).
Analysis in prior years indicated no significant impacts overall on either reading
or math achievement.
- The OSP had a positive impact overall on parents' reports of school satisfaction
and safety (figures 3 and
4), but not on students' reports (figures
3 and 4). Parents were more satisfied
with their child's school (as measured by the percentage giving the school a grade
of A or B) and viewed their child's school as safer and more orderly if the child
was offered a scholarship. Students had a different view of their schools than did
their parents. Reports of safety and school climate were comparable for students
in the treatment and control groups. Overall, student satisfaction was unaffected
by the Program.
- This same pattern of findings holds when the analysis is conducted to determine
the impact of using a scholarship rather than being offered a
scholarship. Fourteen percent of students in our impact sample who were
randomly assigned by lottery to receive a scholarship and who responded to year
3 data collection chose not to use their scholarship at any point over the 3-year
period after applying to the Program.1 We
use a common statistical technique to take those "never users" into account; it
assumes that the students had zero impact from the OSP, but it does not change the
statistical significance of the original impact estimates. Therefore, the positive
impacts on reading achievement, parent views of school safety and climate, and parent
views of satisfaction all increase in size, and there remains no impact on math
achievement and no overall impact on students' perceptions of school safety and
climate or satisfaction from using an OSP scholarship.
- The OSP improved reading achievement for 5 of the 10 subgroups examined.2 Being offered or using a scholarship led to higher
reading test scores for participants who applied from schools that were not classified
as "schools in need of improvement" (non-SINI). There were also positive impacts
for students who applied to the Program with relatively higher levels of academic
performance, female students, students entering grades K-8 at the time of application,
and students from the first cohort of applicants. These impacts translate into 1/3
to 2 years of additional learning growth. However, the positive subgroup reading
impacts for female students and the first cohort of applicants should be interpreted
with caution, as reliability tests suggest that they could be false discoveries.
- No achievement impacts were observed for five other subgroups of students,
including those who entered the Program with relative academic disadvantage.
Subgroups of students who applied from SINI schools (designated by Congress as the
highest priority group for the Program) or were in the lower third of the test score
distribution among applicants did not demonstrate significant impacts on reading
test scores if they were offered or used a scholarship. In addition, male students,
those entering high school grades upon application, and those in application cohort
2 showed no significant impacts in either reading or math after 3 years.
Top