Skip Navigation
Evaluation Studies of the National Center for Education Evaluation and Regional Assistance

Implementation and Impact Evaluation of Race to the Top and School Improvement Grants

Contractors: Mathematica Policy Research, Inc.; American Institutes for Research; Social Policy Research Associates

Background/Research Questions:

Race to the Top (RTT) was an ED-sponsored competitive grant program that funded states and districts planning to implement comprehensive education reform in one or more core areas. Since 2010, RTT has funded a general state competition, as well as a state competition focused on early learning and a district competition focused on personalized learning. With funds from the American Recovery and Reinvestment Act of 2009 (ARRA), the general state competition awarded $4 billion to states in support of comprehensive K–12 education reform in six core areas: state capacity, teachers and leaders, standards and assessments, data systems, school turnaround, and charter schools.

The School Improvement Grants (SIG) program was authorized through Title I of the Elementary and Secondary Education Act and provided 3-year awards to support turnaround in the nation's persistently lowest-achieving schools. In fiscal year 2009, the $546 million SIG appropriation was supplemented by $3 billion through ARRA, for a total of $3.5 billion. SIG funds were disbursed to states by formula based on Title I allocations. States then competitively awarded funds to districts applying on behalf of their eligible schools. Schools receiving SIG as part of the ARRA-funded cohorts were required to implement one of four prescriptive intervention models: turnaround, transformation, closure, or restart.

RTT and SIG were signature ED programs in the Obama Administration, and both programs received substantial funding through ARRA. As part of OMB's FY 2010 Evaluation Initiative, IES conducted an impact evaluation of the Race to the Top and School Improvement Grant programs, focusing on the initial general state competition for RTT and the first cohort of SIG schools implementing intervention models beginning in the 2010–11 school year. In consultation with OMB, IES addressed five evaluation questions:

  • Which policies and practices promoted by the RTT program do RTT states report using, and how do they compare to the policies and practices that non-RTT states report using?
  • Is receipt of an RTT grant related to improvement in student outcomes?
  • Are SIG-funded schools using the improvement or turnaround strategies promoted by the four SIG intervention models, and how do they compare to strategies in schools not implementing a SIG-funded intervention model? How are states and districts supporting such efforts?
  • Does receipt of SIG funding to implement a school intervention model have an impact on outcomes for low-performing schools?
  • Is implementation of the four school intervention models related to improvement in outcomes for low-performing schools?

Design:

The RTT sample included all 50 states and DC. Data from interviews with all states and DC informed the first evaluation question. The second evaluation question was addressed using a short interrupted time series design with state-level National Assessment of Educational Progress data comparing, before and after the RTT competition, states that were awarded an RTT grant to states that applied for but were not awarded an RTT grant.

The SIG sample included about 500 schools in 60 districts from 22 states. This sample was purposively selected to support a regression discontinuity design to address the fourth evaluation question, exploiting cutoff rules that states used to identify their persistently lowest-achieving schools as eligible for SIG to implement one of the four intervention models. Data from state and district interviews, as well as school surveys from the SIG sample informed the third and fifth evaluation questions. Student- and school-level achievement data were also collected from administrative records up to the 2012–2013 school year to inform the fourth and fifth evaluation questions.

It was expected that RTT and SIG requirements would be implemented over a number of years. Therefore, the evaluation focused on implementation at the state, district, and school levels in the early years of the study. Additional years of data collection beyond the scope of this contract will likely be needed to fully describe the implementation and impact of these efforts.

Cost/Duration: $15,298,134 over 6 years (September 2010–April 2017)

Current Status:

All data collection and analyses have been completed. Two reports and four evaluation briefs have been released (see below for key findings). A final report for SIG is anticipated in January 2017 and will be announced on http://ies.ed.gov/ncee/.

Key Findings:

A final report entitled Race to the Top: Implementation and Relationship to Student Outcomes was released on October 26, 2016. Using 2013 interview data from all states, this report documents whether states that received an RTT grant used the policies and practices promoted by the program and how that compares to non-grantee states. The report also examines whether receipt of an RTT grant was related to improvements in student outcomes using data from the National Assessment of Educational Progress. Key findings include:

  • In four of six areas examined, 2010 RTT grantees reported using more policies and practices promoted by RTT than states that did not receive a grant: The four areas with differences were standards and assessments, teachers and leaders, school turnaround, and charter schools. The other two areas were state capacity and data systems;
  • 2011 RTT grantees reported using more policies and practices promoted by RTT than states that did not receive a grant in one area, which was teachers and leaders;
  • Across all states, use of RTT-promoted policies and practices were highest in the data systems area and lowest in the teachers and leaders area: States reported using 76 percent of the 8 RTT-promoted practices examined in data systems, but only 26 percent of the 39 practices in teachers and leaders; and
  • The relationship between RTT and student outcomes was not clear: Trends in student outcomes could be interpreted as providing evidence that RTT had a positive effect, a negative effect, or no effect.

This final report follows the release of an initial report on September 29, 2015 entitled Usage of Policies and Practices Promoted by Race to the Top and School Improvement Grants. The initial report used data from a year earlier (spring 2012) to similarly document state and school use of policies and practices promoted by RTT and SIG, and whether grantees were more likely to use these policies and practices than non-grantees.

Four companion evaluation briefs have also been released, which provide a more focused look at specific topics covered in the reports. The first was released on January 8, 2014 and examined the extent to which low-performing schools received greater operational authority, support, and monitoring with SIG. The second was released on April 30, 2014 and examined the extent to which state requirements for teacher evaluation policies aligned with those promoted by RTT. The third was released on October 28, 2014 and examined whether low-performing schools adopted the various improvement practices promoted by SIG. The fourth was released on May 5, 2015 and described states' capacity to support the improvement efforts of their low-performing schools.