Skip Navigation
Print Evaluations

Implementation and Impact Evaluation of Race to the Top and School Improvement Grants

Contract Information

Current Status:

This study has been completed.

Duration:

September 2010 – April 2017

Cost:

$15,298,134

Contract Number:

ED-IES-10-C-0077

Contractor(s):

Mathematica Policy Research
American Institutes for Research
Social Policy Research Associates

Contact:

The American Recovery and Reinvestment Act of 2009 (ARRA) was a $831 billion economic stimulus package that included $100 billion for states and districts to secure teachers' jobs and promote innovation in schools. Among the federal education programs that ARRA provided funding for were Race to the Top (RTT, $4 billion) and School Improvement Grants (SIG, $3 billion).

RTT was a competitive discretionary grant program announced in 2009 that funded states and districts planning to implement comprehensive education reform in one or more core areas. Beginning in 2010, RTT funded a general state competition, as well as a state competition focused on early learning and a district competition focused on personalized learning. The general state competition awarded $4 billion to states in support of comprehensive K–12 education reform in six core areas: state capacity, teachers and leaders, standards and assessments, data systems, school turnaround, and charter schools.

Authorized by Title I, Section 1003(g) of the No Child Left Behind Act of 2001, SIG sought to help improve the nation's persistently lowest-achieving schools. The SIG program received a large influx of more than $7 billion between 2009 and 2016. The specific amount that states received from the U.S. Department of Education was based on Title I formulas. States then competitively distributed the funds to districts applying on behalf of their eligible schools. The first cohort of schools to receive this influx of SIG funds were required to implement one of four prescribed intervention models over three years (2010–11 to 2012–13): transformation, turnaround, restart, or closure.

Given that ARRA made a substantial investment in both the RTT and SIG programs, it is of interest to better understand how the programs were implemented and what impacts they may have had on student outcomes. This evaluation was conducted as part of the Office of Management and Budget's Evaluation Initiative for Fiscal Year 2010. The evaluation focused on the initial general state competition for RTT (with awards in 2010 and 2011) and the first cohort of SIG schools implementing intervention models between the 2010–11 and 2012–13 school years, since these were the parts of the RTT and SIG programs that were primarily funded by ARRA.

  • Which policies and practices promoted by the RTT program did RTT states report using, and how did they compare to the policies and practices that non-RTT states reported using?
  • Did receiving an RTT grant relate to improvements in student outcomes?
  • Were SIG-funded schools using the improvement practices promoted by the four SIG models, and how did they compare to the practices in schools not implementing a SIG-funded model?
  • Did receiving SIG funding to implement an intervention model have an impact on student outcomes?
  • Did schools that implemented a particular SIG model experience greater gains in student outcomes than schools that implemented one of the other SIG models?

The RTT sample included all 50 states and DC. Data from spring 2012 and spring 2013 interviews with all states and DC informed the first evaluation question. The second evaluation question was addressed using a short-interrupted time series design with existing National Assessment of Educational Progress (NAEP) data from 2003 to 2015 for reading and math in grades 4 and 8. State-level NAEP scores were compared both before and after the RTT competition (prior to 2010 vs. after 2011), and between states that were awarded an RTT grant and states that applied for but were not awarded an RTT grant (RTT vs. non-RTT states).

The SIG sample included about 500 schools (both SIG schools and other low-performing schools that did not receive SIG) from 60 districts and 22 states. This sample was purposively selected to support a regression discontinuity design to address the fourth evaluation question. Schools whose performance was just low enough to qualify for SIG were compared to otherwise similar schools whose performance was just high enough to not qualify for SIG. Spring 2012 and spring 2013 data from state interviews, district interviews, and school surveys of the SIG sample informed the third and fifth evaluation questions. Student- and school-level math and reading achievement data in all tested grades from the 2009–10 to 2012–13 school years, as well as high school graduation and college enrollment rates (to the extent available), were also collected from administrative records to inform the fourth and fifth evaluation questions.

The final report for RTT, titled Race to the Top: Implementation and Relationship to Student Outcomes, along with an interactive exploration of findings, was released in October 2016.

The final report for SIG, titled School Improvement Grants: Implementation and Effectiveness, along with an interactive exploration of findings, was released in January 2017.

Other publications from this study are listed below.

Reports on the Implementation of RTT and SIG in Spring 2012

Briefs on Specific Implementation Topics

Race to the Top

  • In spring 2013, 2010 RTT grantees reported using more policies and practices promoted by RTT than states that did not receive a grant in four of six core RTT reform areas. The four areas with differences were standards and assessments, teachers and leaders, school turnaround, and charter schools. The other two areas were state capacity and data systems.
  • In spring 2013, 2011 RTT grantees reported using more policies and practices promoted by RTT than states that did not receive a grant in one area, which was teachers and leaders.
  • In spring 2013 across all states, use of RTT-promoted policies and practices were highest in the data systems area and lowest in the teachers and leaders area: States reported using 76 percent of the 8 RTT-promoted practices examined in data systems, but only 26 percent of the 39 practices in teachers and leaders.
  • The relationship between RTT and student outcomes was not clear. Trends in student outcomes could be interpreted as providing evidence that RTT had a positive effect, a negative effect, or no effect.

School Improvement Grants

  • In spring 2013, SIG schools implementing one of the four models (transformation, turnaround, restart, or closure) reported using more practices promoted by SIG than other schools. SIG schools reported using an average of 23 out of 35 practices examined, whereas other schools reported using 20 practices.
  • In spring 2013 across all schools, use of SIG-promoted practices was highest in the area of comprehensive instructional reform strategies and lowest in the area of operational flexibility and support. Schools reported using 89 percent of the 8 SIG-promoted practices examined in the comprehensive instructional reform strategies area, but only 43 percent of the 2 practices in operational flexibility and support (the other two areas examined were increasing teacher and principal effectiveness, and increasing learning time and creating community-oriented schools).
  • Implementing any of the four SIG models did not have significant impacts on math or reading test scores, high school graduation, or college enrollment.
  • In elementary grades, student achievement gains did not differ across the four SIG models. In secondary grades, the turnaround model was associated with larger achievement gains than the transformation model.