Characteristics of Program Applicants

In determining and interpreting program effectiveness later on, it is important to know how well the program is targeted to the disadvantaged families who are the focus of the program and, beyond that, what types of families and students apply, win scholarships in the lottery, and choose to use them to enroll in a private school. It is also useful to identify the extent to which public and private schools in the District are experiencing a significant loss or gain of students due to the first year of program implementation, because that information provides the foundation for our later examination of the impact of the program on DC schools.

Public School Applicants Compared to Similar Public School Students in DC

There are several reasons to examine the eligible public school applicants to the DC Opportunity Scholarship Program in relation to other DCPS students. Most clearly, the comparison provides a context for considering the kinds of students who might be attracted to the program in future years in the District or to a similar program in other locations. The income eligibility criteria for the program—family income within 185 percent of the federal poverty line—matches up quite closely with eligibility for the federal free or reduced-price lunch (FRL) program. Using this income indicator to compare public school program applicants to similarly low-income DCPS nonapplicants, we find some differences and similarities (Table ES–3).

• Program applicants scored somewhat higher on reading and mathematics accountability tests than the nonapplicants.


• Applicants were more likely than nonapplicants to be African American and less likely to be Hispanic.


• The two groups were similar regarding special education enrollment, gender, and enrollment in the FRL program (the latter by design).

Similar patterns are evident when we use a subset of the applicants—just the program participants—to compare to economically disadvantaged DCPS students in the same grades, as the program statute requires for performance reporting. ⁴

Applicants in the Impact and Non-Impact Samples

While it is important to examine who applies to the DC Opportunity Scholarship Program, for the evaluation it is equally critical to assure that the applicants who will be the focus of the impact analysis—the applicants who were randomly assigned to treatment and control groups randomly assigned to treatment by the lottery—are similar prior to the beginning of the program. It is on this similarity between the groups, not only in characteristics easily measured but also in those not observed, that the scientific benefits of the randomized control trial (RCT) approach rests. We find no statistically significant differences between the two groups on any major educational or family background measures, confirming that the assignment lottery was conducted appropriately.

It is also useful to review how the characteristics of the impact sample differ from the characteristics of students who applied but will not be included in the analysis of program effectiveness (the "non-impact sample" of applicants). The existence of many significant differences between the impact and non-impact samples limits extrapolation of the results of the impact analysis to characterize overall program impact. There are a few differences between the two groups. Compared to their non-impact sample counterparts, members of the impact sample: (1) scored higher in reading in grades 9 through 12, (2) are more likely to have a learning disability, and (3) are less likely to be of Hispanic ethnicity.

Applicants by Type of Previous School

The schools students previously attended may be associated with students’ educational or background characteristics, their parents’ attitudes, and ultimately the extent to which the program is effective for students seeking scholarships. Eligible applicants to the program came from four different types of schools. Four percent came from SINI-designated public schools, 54 percent from non-SINI regular public schools, 14 percent from public charter schools, and 28 percent from various private schools. The most important differences among the applicants include the following:

• Among high school applicants, SINI public applicants scored lower in reading, whereas non-SINI public applicants scored higher.⁵


• The average family income of all applicants was $18,742, with SINI and non-SINI public applicants reporting somewhat lower incomes and private school applicants reporting somewhat higher incomes.


• The mothers of applicants reported an average of almost 13 years of formal education. The mothers of non-SINI public applicants reported slightly lower levels of education, and the mothers of private school applicants slightly higher levels.

Applicant Response Rates Among Public and Private Schools

A central question in the debate surrounding school choice—and one of the topics the statute requires the evaluation to address—is whether a scholarship program has an impact on the larger public and private school systems. Such "systemic effects" could take place if significant percentages of students in the public school system or in specific schools apply for, receive, and use scholarships to transfer to private schools. With regard to the public schools, the DC Opportunity Scholarship Program could have either positive or negative effects. One theoretical argument suggests that scholarship programs will divert funding and the most motivated students from public schools to private schools, leaving the public school system with fewer resources with which to educate the remaining student population. Another theory is that schools behave in a manner similar to firms and will respond to competition by becoming more efficient. In the case of schools, it is the risk of losing students and subsequently funding that may provide an impetus for public schools to provide better services and produce better student outcomes. Private schools in jurisdictions with greater school choice may face similar incentives to improve or expand in order to retain as many of their current students and attract more.

To provide the basis for a later analysis of the effects of the DC Opportunity Scholarship Program on schools, we first need to describe the extent to which public schools have so far been affected by program applications and scholarship users—a possible predictor of the level of competitive pressure the public schools may experience. The school-level scholarship application and use rates this first year suggest that relatively few public schools have experienced a significant loss of students as a result of the scholarship program (Table ES-4).

• Over one-quarter of the public schools in the District experienced no student losses due to the program.


• Another 56 percent of DCPS schools had program-related transfers out that totaled less than 2 percent of their student populations.


• Seventeen percent of District schools lost about 2 to 4 percent of their students.


• Finally, 2 percent of the public schools in the District experienced more significant student transfers of over 4 percent under the program.

In contrast, a similar analysis of the participating private schools suggests that students using DC Opportunity Scholarships make up a significant share of their enrollments. In more than one-quarter of those private schools, nearly 20 percent of their students are using Opportunity Scholarships; in another 37 percent, scholarship students make up between 5 and 20 percent of their student populations in 2004-05.

Top