The Act requires that this 5-year scholarship pilot Program be rigorously evaluated by an independent research team, using the "strongest possible research design for determining the effectiveness" of the Program and addressing a specific set of student comparisons and topics (Section 309):
3 For examples, see the What Works Clearinghouse, WWC Study Review Standards, 7 (http://www.whatworks.ed.gov/ reviewprocess/studv_standards_final.pdf); Thomas D. Cook and Monique R. Payne, "Objecting to the Objections to Using Random Assignment in Educational Research," in Evidence Matters: Randomized Trials in Education Research, eds. Frederick Mosteller and Robert Baruch (Washington, DC: Brookings, 2002).
4 DCPS students who did not apply to the scholarship Program are likely to be quite different from those who applied and are participating in the OSP-in ways we can observe and ways we cannot. Comparing outcomes between participants and nonapplicants is, therefore, not a reliable measure of Program effects.
5 Automatic scholarship awards were given only in the first year of Program implementation to all students applying from public schools designated "in need of improvement" under the 2002 reauthorization of the Elementary and Secondary Education Act and to all public school applicants entering grades K–5 that were not oversubscribed and therefore not subject to award by lottery.
6 The "recipient sample" is different from the "impact sample," which is limited to public school applicants who were subject to scholarship award by lottery and thus were randomly assigned to the "treatment" (scholarship) or "control" (nonscholarship) group.
7 See Section 309(a)(3)(B) for the provision that stipulates that the evaluation use the same assessment as DCPS administered to public school students in the first year the OSP was operating. This requirement was intended to ensure that the impact analysis could be based on a consistent measure of student achievement and not subject to changes in the key outcome measure throughout the evaluation period.