Impact Evaluation of Race to the Top and School Improvement Grants

Contractors: Mathematica Policy Research, Inc.; American Institutes for Research; Social Policy Research Associates

Background/Research Questions:

Race to the Top (RTT) is an ED-sponsored competitive grant program to states that has committed $4 billion through ARRA to support comprehensive K–12 education reform in four major areas: teachers and leaders, standards and assessments, data systems, and school turnaround. School Improvement Grants (SIG) are authorized and funded through Title I of the Elementary and Secondary Education Act, with a supplement through ARRA, for a total of $3.5 billion. Awards are given to SEAs in proportion to Title I funding, and schools receiving SIG must implement one of four school reform models: turnaround, transformation, closure, or restart. Although the award mechanisms and the state and district context of grantees under the two programs differ substantially, grantees under both programs are required to implement the same four turnaround models in their lowest-achieving schools. As part of OMB's FY 2010 Evaluation Initiative, IES proposed to conduct an impact evaluation of the Race to the Top and School Improvement Grant programs. In consultation with OMB, IES is planning to address four complementary evaluation questions:

• How are RTT and SIG implemented at the state, district, and school levels with respect to standards and assessments, data systems, teachers and leaders, school turnaround models including charter schools, and overall state capacity?
• Are RTT reforms related to improvement in student outcomes?
• Does receipt of RTT and/or SIG funding to implement a school turnaround model have an impact on outcomes for lowest-achieving schools?
• Is implementation of the four school turnaround models, and strategies within those models, related to improvement in outcomes for lowest-achieving schools?

Design:

States identified their lowest-achieving schools as eligible for funding to implement a school turnaround model (STM). To the extent that a consistent measure(s) and/or rules were used across the schools within states or districts, the study plans to use a regression discontinuity design to estimate the impact on student outcomes of receiving RTT and/or SIG funding to implement a STM. This sample will include about 134 districts from 30 states and a total of 1,200 STM and non-STM comparison schools. The RTT sample will include all 50 states and DC for the implementation study component, and will utilize an interrupted time series design with state-level National Assessment of Educational Progress data to analyze the relationship between RTT and student outcomes. Three years of data collection are planned including state interviews, district interviews, school surveys, and extant student achievement information.

It is expected that RTT and SIG requirements will be implemented over a number of years. Therefore, the evaluation plans to focus on implementation at the state, district, and school levels in the early years of the study. We also anticipate that additional years of data collection beyond the scope of this contract will be needed to fully describe the implementation and impact of these efforts.

Cost/Duration: $18,606,416 over 5 years (September 2010–September 2015)

Current Status:

The contract was awarded in September 2010 and is currently in the design and recruitment phases. Data collection is scheduled to begin in March 2012, and the first report is scheduled to be released in March 2014.