Skip Navigation
archived information
Stay Up-to-Date:
Skip Navigation

REL Southwest Ask A REL Response

School Improvement:

Effects of Whole School Reform

November 2019

Question:

What does the research say are the effects of whole school reform actions such as closing and reopening a school under district or charter management, school closure with student reassignment, or starting a brand new charter or district school?

Response:

Print-friendly version (568 KB) PDF icon

Thank you for the question you submitted to our REL Reference Desk. We have prepared the following memo with research references to help answer your question. For each reference, we provide an abstract, excerpt, or summary written by the study’s author or publisher. Following an established Regional Educational Laboratory (REL) Southwest research protocol, we conducted a search for research reports as well as descriptive study articles on the effects of district-level whole school reform actions involving closing and restarting schools.

We have not evaluated the quality of references and the resources provided in this response. We offer them only for your reference. Also, we searched the references in the response from the most commonly used resources of research, but they are not comprehensive, and other relevant references and resources may exist. References provided are listed in alphabetical order, not necessarily in order of relevance. We do not include sources that are not freely available to the requestor.

Research References

Ayscue, J. B., & Siegel-Hawley, G. (2019). Magnets and school turnarounds: Revisiting policies for promoting equitable, diverse schools. Education Policy Analysis Archives, 27(72). https://eric.ed.gov/?id=EJ1219388

From the ERIC abstract: “This case study examines how magnet school and school turnaround processes can work together to promote desegregation and improvement. Based on cross-case analysis of three magnet schools undergoing turnarounds, this study draws on data from the 2010 federal Magnet Schools Assistance Program grant and qualitative fieldwork through observations, interviews, and focus groups. In academically struggling schools with high concentrations of students of color and low-income students, successful magnet turnarounds involve changes across many aspects of the schools. While the local context is essential for shaping the magnet turnaround process, these three schools reveal common ways in which participants viewed their schools as successful turnarounds, the elements that supported success, and the challenges that magnets undergoing a turnaround are likely to face. Participants’ perceptions of a successful turnaround were based on increasing family interest and increasing racial and economic diversity, as well as improvements in curriculum and instruction, school culture, and academic achievement. This study helps broaden our definition of a school turnaround beyond higher test scores and reminds us of the origins of the concept, which revolved around desegregation. Lessons from the sites suggest that rather than closing underperforming or under-enrolled schools, districts should consider magnet schools as a turnaround approach.”

Basileo, L. D., & Toth, M. (2017, April). Turning around chronically low-performing schools. AERA Online Paper Repository, Paper presented at the Annual Meeting of the American Educational Research Association, San Antonio, Texas. https://eric.ed.gov/?id=ED593311

From the ERIC abstract: “The study used tailored interventions for ten schools which were deemed chronically low performing in Florida. Treatment schools included a majority of students that were minorities, economically disadvantaged and academically struggling or at-risk of failure. The ten schools received a progression of intensive professional development and coaching for principals and teachers. Principals and district leaders received specific and actionable feedback in 30-day cycles to improve teaching and ensure fidelity to standards-based instruction. While teacher and student value-added metrics are not yet available for analysis, preliminary results show that 70% of schools increased their school letter grade. Moreover, three treatment schools increased two letter grades, four schools increased one letter grade and three schools received the same letter grade.”

Champagne, E., & Therriault, S. B. (2018). Leveraging research to improve Massachusetts’s schools. State Education Standard, 18(2), 24–27. https://eric.ed.gov/?id=EJ1179846

From the ERIC abstract: “The Massachusetts Department of Elementary and Secondary Education (ESE) and the Massachusetts Board of Elementary and Secondary Education are deeply committed to ensuring that every student in the state has access to a high- quality education, and they have combined this commitment with an openness to innovation. After just one year of School Improvement Grant (SIG) implementation in the state's turnaround schools, Education Commissioner Mitchell Chester in 2011 asked his team to document what was working in the state’s lowest performing schools that had engaged in redesign efforts. In 2014, they distilled their findings into a set of turnaround best practices. Supported by this research, ESE began to align its assistance efforts with these turnaround practices. The department took four key steps: (1) It revised the required turnaround planning template to allow schools and districts to choose evidence-based strategies that fit their contexts, so long as they were aligned with the framework of practices; (2) It aligned its application and scoring rubric for competitive federal SIG funding with the revised turnaround plan and the research results from successful schools and in so doing raised the bar for all SIG-funded schools, which could no longer focus on only one or two strategies of varying quality; (3) It changed its process for monitoring progress, adding a turnaround practices and indicators rubric to assess implementation of each of the best practices at its turnaround schools; and (4) It focused its direct assistance toward supporting better implementation of the turnaround practices in the field, aligned with the needs identified through the monitoring process.”

de la Torre, M., Allensworth, E., Jagesic, S., Sebastian, J., Salmonowicz, M., Meyers, C., et al. (2012). Changes in student populations and teacher workforce in low-performing Chicago schools targeted for reform. Issues & answers (REL 2012-123). Washington, DC: U.S. Department of Education, Institute of Education Sciences, National Center for Education Evaluation and Regional Assistance, Regional Educational Laboratory Midwest. https://eric.ed.gov/?id=ED531351

From the ERIC abstract: “‘Turning around’ chronically low-performing schools is of increasing interest to educators and policymakers, as highlighted by the U.S. Department of Education’s (2010) recent call to rapidly improve the nation’s 5,000 lowest performing schools. Yet there is little rigorous research on changes in student populations and teacher workforce in schools undergoing interventions to improve low-performing schools. To fill this gap, this study examines turnaround intervention models intended to rapidly improve student performance in chronically low-performing schools in the Chicago Public Schools district. It analyzes the changes in student populations and teacher workforce in 31 public schools in Chicago selected for district-led reform interventions for chronically low-performing schools over 1997-2010. Two research questions guided the study: (1) Did the characteristics of students change in the intervention schools?; and (2) Did the characteristics of teachers change in the intervention schools? For the first research question, descriptive analyses compared students in the school the fall before the intervention with students in the same grades in the fall after the intervention began. For the second, descriptive analyses compared the teacher workforce in these schools for the same periods. These descriptive analyses show school-by-school changes in students and teachers organized around the intervention models. The analyses are based on the entire population of students and teachers at each school and are not statistical estimates. Comparing student enrollment the fall before the intervention and the fall after the intervention shows that: (1) Twenty-three of 31 schools served fewer students by grade after the intervention, with five schools serving at least a quarter fewer students. Four of the schools with the largest declines in enrollment were part of the closure and restart model; (2) Except for schools in the closure and restart model, schools reenrolled 55-89 percent of students eligible to reenroll. The rates were similar to reenrollment rates in the years before intervention; (3) Schools in the closure and restart model reenrolled 0-47 percent of students eligible to reenroll. Schools in this model were closed for one or two years before opening again, did not serve all the same grade levels when they reopened, and held citywide enrollment lotteries, which made it difficult for students to reenroll; and (4) The composition of the student body—in race/ethnicity, socioeconomic status, and special education status—in intervention schools was largely similar before and after the interventions in all models except for the closure and restart model. In that model, schools after intervention served a larger percentage of economically advantaged students and of students with higher prior achievement levels, and smaller percentages of special education students and of students residing in the neighborhood near the school. Comparing the teacher workforce the year before the intervention and the year after the intervention shows that: (1) The extent of teacher rehiring varied with the intervention model. Schools in the reconstitution model rehired 42-66 percent of teachers, and schools in the STSP model retained 44-80 percent. Schools in the closure and restart, AUSL, and OSI models rehired just 0-24 percent of teachers; and (2) In all intervention models, the teacher workforce was more likely to be White, younger, and less experienced and more likely to have provisional certification after intervention than before it.”

Dragoset, L., James-Burdumy, S., Hallgren, K., Perez-Johnson, I., Herrmann, M., Tuttle, C., et al. (2015). Usage of practices promoted by School Improvement Grants (NCEE 2015- 4019). Washington, DC: National Center for Education Evaluation and Regional Assistance, Institute of Education Sciences, U.S. Department of Education. https://eric.ed.gov/?id=ED559928

From the ERIC abstract: “The American Recovery and Reinvestment Act of 2009 injected $7 billion into two of the Obama administration's signature competitive education grant programs: Race to the Top (RTT) and School Improvement Grants (SIG). While RTT focused on state policies and SIG focused on school practices, both programs promoted related policies and practices, including an emphasis on turning around the nation's lowest-performing schools. Despite the sizable investment in both of these programs, comprehensive evidence on their implementation and impact has been limited to date. This report focuses on two implementation questions: (1) Do states and schools that received grants actually use the policies and practices promoted by these two programs? (2) Does their usage of these policies and practices differ from states and schools that did not receive grants? Answers to these questions provide context for interpreting impact findings that will be presented in a future report. The second volume of this report details our SIG findings, which are based on spring 2012 surveys of approximately 470 schools in 60 districts and 22 states. Key findings include: (1) Schools implementing a SIG-funded model reported using more practices promoted by SIG than schools not implementing such models in all four areas examined: comprehensive instructional reforms, teacher and principal effectiveness, learning time and community- oriented schools, and operational flexibility and support; (2) Across all schools, usage of practices promoted by SIG was highest in the comprehensive instructional reforms area (90 percent of practices examined) and lowest in the operational flexibility and support area (46 percent of practices examined); and (3) There were no differences between schools implementing a SIG-funded model and schools not implementing one in usage of ELL-focused practices promoted by SIG. Schools with higher percentages of ELLs used more ELL-focused practices than schools with lower percentages of ELLs, but there were no differences in usage between schools with higher and lower ELL/non-ELL achievement gaps.”

Dragoset, L., Thomas, J., Herrmann, M., Deke, J., James-Burdumy, S., Graczewski, C., et al. (2017). School Improvement Grants: Implementation and effectiveness (NCEE 2017- 4013). Washington, DC: National Center for Education Evaluation and Regional Assistance, Institute of Education Sciences, U.S. Department of Education. https://eric.ed.gov/?id=ED572215

From the ERIC abstract: “In response to the recession that began in 2007, the U.S. Congress passed, and President Barack Obama signed into law, the American Recovery and Reinvestment Act of 2009 (Pub. Law 111-5). At an estimated cost of $831 billion, this economic stimulus package sought to save and create jobs, provide temporary relief to those adversely affected by the recession, and invest in education, health, infrastructure, and renewable energy. States and school districts received $100 billion to secure teachers' jobs and promote innovation in schools. This funding included $3 billion for School Improvement Grants (SIG), one of the Obama administration’s signature programs and one of the largest federal government investments in an education grant program. The SIG program awarded grants to states that agreed to implement one of four school intervention models—transformation, turnaround, restart, or closure-in their lowest-performing schools. Each of the models prescribed specific practices designed to improve student outcomes, including outcomes for high-need students such as English language learners (ELLs) (U.S. Department of Education 2010a). Given the importance of the SIG program and sizable investment in it, the Institute of Education Sciences (IES) commissioned this evaluation to focus on four primary questions: (1) Did schools implementing a SIG-funded model use the improvement practices promoted by SIG, and how did that compare to use of those practices by schools not implementing a SIG- funded model?; (2) Did use of SIG-promoted practices include a focus on ELLs, and did that focus on ELLs differ between schools implementing a SIG-funded model and schools not implementing one?; (3) Did receipt of SIG funding to implement a school intervention model have an impact on outcomes for low-performing schools?; and (4) Was the type of school intervention model implemented related to improvement in outcomes for low-performing schools? This is the final report for this evaluation of SIG. This final report builds on the earlier briefs and report by including an additional year of data (spring 2013) and by examining whether receipt of SIG funding had an impact on student outcomes. The findings in this report suggest that the SIG program did not have an impact on the use of practices promoted by the program or on student outcomes (including math or reading test scores, high school graduation, or college enrollment), at least for schools near the SIG eligibility cutoff.”

Flowers, N., Begum, S., Carpenter, D. M. H., & Mulhall, P. F. (2017). Turnaround success: An exploratory study of three middle grades schools that achieved positive contextual and achievement outcomes using the Schools to Watch i3 project. RMLE Online: Research in Middle Level Education, 40(8), 1–14. https://eric.ed.gov/?id=EJ1157096. Retrieved from https://www.researchgate.net/publication/320293334_Turnaround_Success_An_Exploratory_Study_of_Three_Middle_Grades_Schools_that_Achieved_Positive_Contextual_and_Achievement_Outcomes_Using_the_Schools_to_Watch_i3_Project

From the ERIC abstract: “This article presents the results of a study examining the highest implementers of the Schools to Watch (STW): School Transformation Network Project, and the impact the project had on their teaching/learning practices and outcomes. The project was funded by the U.S. Department of Education's Investing in Innovation (i3) grant program and was designed to improve the educational practices, experiences, and outcomes of 18 low-performing middle grades schools in California, Illinois, and North Carolina by offering a whole school reform model with a multilayered system of support. The framework for the project was the National Forum to Accelerate Middle- Grades Reform's STW criteria, a set of strategies and practices for middle grades schools. The data presented were collected as part of the project's evaluation which used a quasi- experimental design. Data are from the Forum's STW Rubric and the Center for Prevention Research and Development's (CPRD) Self-Study Teacher Survey from project schools, as well as student achievement test scores from project and comparison schools. The findings showed that the highest implemented STW Project schools achieved higher levels of implementation of the STW criteria, teacher collaboration, leadership practices, and best middle grades instructional practices, and showed significant improvement in mathematics achievement.”

Gandhi, A. G., Slama, R., & Park, S. J. (2016, March). Focusing on the whole student: An evaluation of Massachusetts’ Wraparound Zones Initiative. Paper presented at the Spring 2016 Conference of the Society for Research on Educational Effectiveness, Washington, DC. https://eric.ed.gov/?id=ED567241

From the ERIC abstract: “Over the past twenty years, efforts to turn around low- performing schools have increasingly become a central component of federal and state education policy agendas. The purpose of the study was to evaluate the impact of the Wraparound Zones Initiative (WAZ), a program supported by the Massachusetts Department of Elementary and Secondary Education (ESE), on student outcomes including achievement, attendance, retention, and suspension. The study was conducted as part of a multi-year mixed-methods, formative and summative evaluation of WAZ that ESE commissioned from American Institutes for Research (AIR). The setting consisted of districts and schools in Massachusetts that had been identified by the state as chronically underperforming and in need of state intervention. The sample for this study was drawn from students in Cohort 1 and Cohort 2 WAZ schools serving elementary and/or middle grades, plus students in a set of matched non-WAZ comparison schools. The WAZ Initiative is designed to create coordinated district systems that allow schools to proactively and systematically address students’ nonacademic needs. Comparative interrupted time series (CITS) design was used to measure the impact of receiving a WAZ grant on student outcomes, including student achievement, attendance, retention, and suspension. Overall, students in WAZ schools performed better on the Massachusetts Comprehensive Assessment System (MCAS) English language arts (ELA) and mathematics assessments as compared with students in comparison schools, when considering prior achievement trends. Effects were statistically significant after the second and third years of WAZ implementation for ELA, and after the second year for mathematics. This research study is significant in that it demonstrates that a program focused on student support and social-emotional learning can have an impact on student achievement, and can be an integral component of overall school turnaround strategy.”

Garrison-Mogren, R., & Gutmann, B. (2012). State and district receipt of Recovery Act funds—A report from charting the progress of education reform: An evaluation of the Recovery Act’s role (NCEE 2012-4057). Washington, DC: National Center for Education Evaluation and Regional Assistance, Institute of Education Sciences, U.S. Department of Education. https://eric.ed.gov/?id=ED535324

From the ERIC abstract: “The American Recovery and Reinvestment Act (ARRA or the Recovery Act) of 2009 provided an unprecedented level of funding designed to ‘stimulate the economy in the short-term and invest wisely, using these funds to improve schools, raise achievement, drive reforms and produce better results for children and young people for the long-term health of our nation.’ The distribution of Recovery Act funds was intended to reflect these multiple goals. Nearly $97.4 billion were allocated to the U.S. Department of Education (ED), of which $70.6 billion were awarded by ED for primary and secondary (K-12) education through existing and new federal programs. These funds were distributed to states and districts using formulas based primarily on population and student poverty and through competitive grants. Consistent with its emphasis on transparency, the Recovery Act also included extensive reporting requirements for the receipt and use of Recovery Act funds. This report brings together publicly available information about Recovery Act education grants—all awarded by September 30, 2010— and the sub-grants made by grant recipients as of December 31, 2010. It examines (1) how much states and districts received from the Recovery Act and its different programs; and (2) whether and how the distribution of funds varied by selected characteristics of the recipient states and districts. This information lays the groundwork for ED's multi-year evaluation, ‘Charting the Progress of Education Reform: An Evaluation of the Recovery Act's Role.’ The evaluation examines the implementation of K-12 education reforms promoted by the Act across states, school districts, and schools. Key findings from this examination reveal that: (1) the Recovery Act provided an average of $1,396 per pupil for K-12 programs; (2) the Recovery Act K-12 funding to individual states ranged from $1,063 to $3,632 per pupil; (3) on average, 81 percent of Recovery Act K-12 funding was awarded to local education agencies (LEAs), either through sub-grants from states or through direct grants from ED. In total, 93 percent of all school districts in the nation received Recovery Act funds from at least one program; and (4) high-need school districts—defined as those with the highest rates of child poverty as well as those with the lowest student achievement—received considerably more funding per pupil than did districts with less need.”

Golden, L., Harris, B., Mercado-Garcia, D., Boyle, A., Le Floch, K. C., & O’Day, J. (2014). A focused look at schools receiving School Improvement Grants that have percentages of English language learner students. NCEE Evaluation Brief (NCEE 2014-4014). Washington, DC: National Center for Education Evaluation and Regional Assistance, Institute of Education Sciences, U.S. Department of Education. https://eric.ed.gov/?id=ED544789

From the ERIC abstract: “The Study of School Turnaround examines the improvement process in a purposive sample of 35 case study schools receiving federal funds through the School Improvement Grants (SIG) program over a three-year period (2010-11 to 2012-13 school years). This evaluation brief focuses on 11 of these SIG schools with high proportions of English Language Learner (ELL) students (a median of 45 percent ELLs). Three key findings that emerged from the ELL case study data collected in fall 2011 include: (1) Although all 11 schools reported providing specialized supports for ELL students, the schools' approaches to improvement during the initial phase of SIG appeared to include only moderate or limited attention to the unique needs of ELLs; (2) District and school administrators perceived challenges related to teachers' expertise and skills in meeting the unique needs of ELLs; however, teachers' perceptions of their own capacity were more mixed. The capacity of the schools' district offices to support ELLs appeared to vary as well, with two small districts reporting no district-level staff with ELL training or experience and seven larger districts reporting district-level English-as-a- second-language (ESL) departments with multiple trained staff members; and (3) Schools that appeared to provide stronger attention to the unique needs of ELLs in their improvement process were more likely to report having school staff dedicated to ELL needs, such as ELL coordinators, ELL coaches, and ESL/bilingual teachers and tutors. Such schools also were more likely to be located in districts that reportedly provided expertise and an explicit focus on ELLs within the context of SIG.”

Herman, R., Dawson, P., Dee, T., Greene, J., Maynard, R., Redding, S., et al. (2008). Turning around chronically low-performing schools: A practice guide (NCEE 2008- 4020). Washington, DC: National Center for Education Evaluation and Regional Assistance, Institute of Education Sciences, U.S. Department of Education. https://eric.ed.gov/?id=ED501241

From the ERIC abstract: “This guide identifies practices that can improve the performance of chronically low-performing schools—a process commonly referred to as creating ‘turnaround schools.’ The four recommendations in this guide work together to help failing schools make adequate yearly progress. These recommendations are: (1) signal the need for dramatic change with strong leadership; (2) maintain a consistent focus on improving instruction; (3) provide visible improvements early in the turnaround process (quick wins); and (4) build a committed staff. The guide includes a checklist showing how each recommendation can be carried out. It uses examples from case studies which illustrate practices noted by schools as having had a positive impact on the school turnaround.”

Herman, R., Graczewski, C., James-Burdumy, S., Murray, M., Perez-Johnson, I., & Tanenbaum, C. (2013). Operational authority, support, and monitoring of school turnaround. NCEE Evaluation Brief. NCEE 2014-4008. Washington, DC: National Center for Education Evaluation and Regional Assistance, Institute of Education Sciences, U.S. Department of Education. https://eric.ed.gov/?id=ED544585

From the ERIC abstract: “The federal School Improvement Grants (SIG) program, to which $3 billion were allocated under the American Recovery and Reinvestment Act of 2009 (ARRA), supports schools attempting to turn around a history of low performance. School turnaround also is a focus of Race to the Top (RTT), another ARRA-supported initiative, which involved a roughly $4 billion comprehensive education reform grant competition for states. Given the size of these federal investments, the Institute of Education Sciences (IES) is conducting a large-scale evaluation of RTT and SIG to better understand the implementation and impacts of these programs. The SIG component, in particular, focuses on a purposive sample of SIG-eligible schools, including: (1) a group of schools that received SIG to implement one of four intervention models specified by the U.S. Department of Education; and (2) a comparison group of schools from the same districts that are not implementing one of these four intervention models with SIG support. Though the results from this evaluation of SIG are not necessarily generalizable to SIG schools nationwide, they are nonetheless important because they add to the limited knowledge base about the implementation and impacts of SIG-funded school turnaround efforts. This brief focuses on the implementation of SIG by examining three interrelated levers for school improvement: (1) school operational authority; (2) state and district support for turnaround; and (3) state monitoring of turnaround efforts. Two appendices present: (1) Race to the Top and School Improvement Grant: Intervention Models as Described by the U.S. Department of Education SIG Guidance (2012); and (2) Table B.1. Characteristics of the State Sample as of 2009-2010 and Table B.2. Characteristics of the District Sample as of 2009-2010.”

Jochim, A., & Opalka, A. (2017). The “City of Firsts” charts a new path on turnaround. Linking state and local school improvement. Seattle, WA: Center on Reinventing Public Education, University of Washington. https://eric.ed.gov/?id=ED574140

From the ERIC abstract: “In 2014, the Springfield Public School district in Massachusetts had tried just about every strategy in the turnaround playbook to improve a set of struggling middle schools, but these efforts failed to generate the desired improvement. In 2015, drawing inspiration from national efforts to infuse schools with enhanced autonomy and accountability, the district voluntarily ceded operational control of six middle school campuses to the newly formed Springfield Empowerment Zone Partnership (SEZP), an independent nonprofit charged with overseeing the turnaround effort. SEZP offers principals freedom from district rules in exchange for increased accountability for results. These changes, along with a new collective bargaining agreement for teachers working in SEZP and new supports for students and principals, represent a marked departure from Springfield’s previous efforts. This case study profiles SEZP, comparing and contrasting it with conventional turnaround strategies such as district-led turnaround, reconstitution, chartering, and state-initiated turnaround. Key findings include: (1) SEZP brought together a package of reforms aimed at generating improvement and a new governance model that gives schools much greater freedom to change without needing to ask permission or fear regulatory second-guessing; (2) SEZP offers a ‘middle way’ between other school turnaround strategies: providing more local participation and less controversy compared to either state takeovers or chartering, and committing more deeply to school autonomy, tailored support, and choice of talent compared to conventional district-led turnarounds; and (3) Thus far, SEZP has seen less controversy and more goodwill than many other turnaround efforts, but whether the strategy results in improved student outcomes remains to be seen.”

Jochim, A., & Pillow, T. (2019). Sustaining improvement after state takeovers: Lessons from New Orleans. Seattle, WA: Center on Reinventing Public Education, University of Washington. https://eric.ed.gov/?id=ED594442

From the ERIC abstract: “In the wake of Hurricane Katrina in 2005, Louisiana’s Recovery School District took over responsibility for most of New Orleans’ public schools. In 2018 the state takeover came to an end. For the first time since Katrina, nearly all of the city’s public schools are in the hands of the Orleans Parish School Board. We set out to understand how the return to local control has shaped the trajectory of education in the city, with an eye toward implications for other states and localities facing similar transitions. We conducted two rounds of interviews, in 2016 and 2018, with local board members, current and former district staff, charter school leaders and board members, community-based advocates, and state officials. Our interviews focused on understanding how the shift to local control has shaped the role of key government officials and their partners in the nonprofit sector, how local leaders are responding to the demands generated by local control, and what concerns people have about the future of education in New Orleans. State and local leaders took numerous steps to ensure that a return to local control did not spell an end to the conditions enabling the academic improvements that occurred under state control. We find, in large part, these preparations had their intended effects. For now, the reforms are insulated from the opposition by unique state legislation designed to protect them, as well as from local leaders who have a stake in sustaining them. But continued progress in New Orleans will hinge upon whether local leaders can assemble support for continued action on low performance and address the emerging challenges facing students, families, and schools. This won’t be easy. The board faces an increasingly raucous opposition, and local leaders have yet to aggressively guide the system of schools in new, productive directions. The wholesale changes that occurred in New Orleans likely could not have happened without state intervention. For other localities worried about sustaining systemic transformations after takeovers end, New Orleans offers key lessons in how proactive policymaking can create a bulwark that safeguards school autonomy and other key features of the reforms and underscores the value of deliberate transfers of knowledge and talent.”

Klute, M., Cherasaro, T., & Apthorp, H. (2016). Summary of research on the association between state interventions in chronically low-performing schools and student achievement (REL 2016-138). Washington, DC: U.S. Department of Education, Institute of Education Sciences, National Center for Education Evaluation and Regional Assistance, Regional Educational Laboratory Central. https://eric.ed.gov/?id=ED565613

From the ERIC abstract: “This report summarizes the research on the association between state interventions in chronically low-performing schools and student achievement. Most of the research focused on one type of state intervention: working with a turnaround partner. Few studies were identified that examined other types of interventions, such as school closure, charter conversion, and school redesign. Most studies were descriptive, which limits the conclusions that can be drawn about the effectiveness of the interventions. Results of studies of turnaround partner interventions were mixed and suggested that student achievement was more likely to improve when particular factors—such as strong leadership, use of data to guide instruction, and a positive school culture characterized by trust and increased expectations for students— were in place in schools. Studies in California examined the Immediate Intervention/Underperforming Schools Program or its successor, the High Priority School Grant Program. Ten studies examined interventions in states other than California. Studies varied somewhat in the details of the interventions studied, including whether additional funding was provided to support implementation of reforms. Unlike interventions in California, studies in other states did not describe school participation in interventions as voluntary.”

Le Floch, K. C., O’Day, J., Birman, B., Hurlburt, S., Nayfack, M., Halloran, C., et al. (2016). Case studies of schools receiving School Improvement Grants: Final report (NCEE 2016-4002). Washington, DC: National Center for Education Evaluation and Regional Assistance, Institute of Education Sciences, U.S. Department of Education. https://eric.ed.gov/?id=ED565615

From the ERIC abstract: “The Study of School Turnaround (SST) examines the change process in a diverse, purposive sample of schools receiving federal School Improvement Grants (SIG) from 2010-11 to 2012-13. With the passage of the American Recovery and Reinvestment Act of 2009 (ARRA), the SIG program underwent three major shifts. First, ARRA boosted total SIG funding in fiscal year 2009 to approximately 6.5 times the original 2009 appropriation through Title I, section 1003(g) of the Elementary and Secondary Education Act (ESEA). SIG funds were distributed to states by formula based on each state's Title I share. States then had to competitively make SIG awards to districts with eligible schools. Second, ARRA targeted funds at only the very worst schools— those that were in the bottom 5 percent of performance and had been low performing for an extended period of time. Third, schools receiving SIG were now required to implement one of four prescriptive intervention models believed to be more aggressive and comprehensive than those generally adopted under prior policies. By increasing the level of funding, better targeting these funds to the persistently lowest-achieving schools, and requiring that schools adopt specific intervention models, the revamped SIG program aimed to catalyze more aggressive efforts to turn around student performance. This report focuses on a small sample of schools receiving SIG over the first three years of the revamped SIG program, from 2010-11 to 2012-13. It presents findings from the study's 25 core sample schools, which were the focus of data collection in spring 2011 and spring 2012, and a subsample of 12 of the 25 schools (collectively referred to as the core subsample), which were selected for data collection in spring 2013 and are the focus of more in-depth analyses looking across all three years of SIG. The findings include: (1) A majority of the 25 core sample schools replaced their principal (21 schools) at least once in the year before SIG (2009-10) or in Year 1 of SIG (2010-11); (2) About half of the 25 core sample schools (12 schools, including 9 turnaround, 2 restart, and 1 transformation) replaced at least 50 percent of their teachers during the 2009-10, 2010-11, or 2011-12 school years; (3) According to teacher survey data, more teachers reported participating in professional learning on math, literacy, and data use than on ELL instruction, special education, or classroom management during Year 2 of SIG (2011-12); (4) Core sample schools reported receiving support from their district (22 of 22 schools) and external support provider(s) (22 of 25 schools), but in some cases, respondents described shortcomings in their district or external support; (5) Among the 12 core subsample schools, those that appeared to engage in more efforts to build human capital in Years 1 and 2 of SIG (7 schools) were more likely to improve their organizational capacity (or sustain their already higher capacity); and (6) Sustainability of any improvements may prove fragile.”

Osborne, D. (2016). Denver expands choice and charters. Education Next, 16(3), 34–43. https://eric.ed.gov/?id=EJ1102667

From the ERIC abstract: “Some of the most dramatic gains in urban education have come from school districts using what is known as a ‘portfolio strategy.’ Under this approach, districts negotiate performance agreements with public schools—traditional, charter, and hybrid models. The arrangement affords school leaders substantial autonomy to handcraft their schools to fit the needs of their students. Districts give parents choices among the schools while working to replicate successful schools and replace failing ones. Many doubt such a strategy is possible with an elected board, because closing schools and laying off teachers triggers fierce resistance. In this article, the author presents how an elected school board in Denver, Colorado, employs portfolio strategy to lift achievement.”

Rosenberg, L., Christianson, M. D., Angus, M. H., & Rosenthal, E. (2014). A focused look at rural schools receiving School Improvement Grants. NCEE Evaluation Brief (NCEE 2014-4013). Washington, DC: National Center for Education Evaluation and Regional Assistance, Institute of Education Sciences, U.S. Department of Education. https://eric.ed.gov/?id=ED544784

From the ERIC abstract: “The Study of School Turnaround is a set of case studies of the school improvement process in a purposive sample of 35 schools receiving federal funds through the School Improvement Grants (SIG) program over a three-year period (school years 2010-11 to 2012-13). This evaluation brief focuses on the nine SIG schools that were in rural areas and how respondents in these schools perceived their rural context to influence specific turnaround activities. Key findings that emerged from the rural case study data collected in spring 2012 include: (1) Although rural SIG schools reported some challenges that nonrural SIG schools have also reported, such as low student motivation and staff morale, the rural schools reported additional challenges resulting from their schools’ remote locations and large catchment areas. For example, respondents reported that these rural characteristics affected the recruitment or retention of teachers and, to a lesser extent, parents’ involvement in the schools. (2) School and district administrators in eight of the nine schools suggested that long teacher commutes or isolated communities posed challenges to recruiting or retaining teachers. To counter these challenges, respondents in two schools reported offering direct support for teacher commutes (for example, gas stipends or vans), and respondents in three schools reported offering signing bonuses to incoming teachers. (3) School and district administrators and teaching staff in the nine schools mentioned multiple factors limiting parent involvement in school-based activities. Respondents from five schools perceived that a lack of access to transportation limited parent involvement, whereas respondents from three schools noted that the distance between schools and parents’ homes was a contributing factor. Four schools focused on hiring or expanding the role of parent liaisons to increase parent involvement.”

Scott, C., & Ostler, N. (2016). Reshaping rural schools in the Northwest Region: Lessons from federal School Improvement Grant implementation (REL 2016-107). Washington, DC: U.S. Department of Education, Institute of Education Sciences, National Center for Education Evaluation and Regional Assistance, Regional Educational Laboratory Northwest. https://eric.ed.gov/?id=ED565803

From the ERIC abstract: “The five states in the Regional Educational Laboratory (REL) Northwest Region have many rural schools that have been designated as in need of improvement. And all five states had rural schools in the first cohort of federal School Improvement Grant (SIG) recipients. To address school improvement, the majority of those schools implemented the transformation model, which requires strategies related to improving instruction, ensuring high-quality staff, and engaging families and communities. REL Northwest Region state and district leaders asked REL Northwest to conduct a study examining the extent to which rural schools across the nation implemented the transformation model, the challenges they experienced, and the technical assistance they received. This report provides information about rural schools using the transformation model. REL Northwest Region leaders may be able to use this study to inform future assistance for their rural schools in need of improvement. A survey was conducted in spring 2014, after most cohort 1 grant activities were complete. The survey respondents included 135 principals (67 percent of the 201 schools surveyed) in rural schools implementing the transformation model. The most salient findings include: (1) Few rural schools fully implemented the SIG transformation model. Only 5 percent of the principals surveyed said their school had fully implemented the 12 transformation strategies that the survey examined. On average, principals said their school had fully implemented 6 of the 12 strategies; (2) More schools implemented strategies related to improving instruction than strategies related to ensuring high-quality staff or engaging families and communities. For example, 77 percent of principals reported that their school had fully implemented the use of student achievement data to inform instructional decisions, whereas 52 percent reported that their school had fully implemented staff evaluation systems that tied evaluation to student achievement, and 40 percent reported that their school had engaged families; (3) More schools reported facing implementation challenges related to ensuring high-quality staff and engaging families and communities than challenges related to improving instruction. For example, almost half (47 percent) of principals reported challenges to rewarding staff financially—a strategy related to ensuring high-quality staff—and about a third (34 percent) reported challenges to engaging families and communities. In contrast, fewer principals (26 percent) reported challenges to expanding learning time to improve instruction; (4) Almost all schools received technical assistance from at least one provider, with districts the most frequently identified provider. Most principals (93 percent) reported that their school had received technical assistance from at least one provider for at least one of the transformation strategies examined in the survey. More principals reported that their school had received this assistance from their district (91 percent) than from the state (70 percent), a university (19 percent), or another type of organization (42 percent); and (5) The more strategies for which principals reported facing challenges, the fewer strategies they reported their school had fully implemented. When principals reported challenges with three or more strategies, they also reported that their school had fully implemented an average of only 5.2 strategies. In contrast, when principals reported challenges with fewer than three strategies, they reported that their school had fully implemented an average of 7.5 strategies. The more strategies for which principals reported receiving technical assistance, the more strategies they reported that their school had fully implemented. When principals reported receiving technical assistance for more than 7 strategies, they also reported that their school had fully implemented an average of 7.2 strategies. In contrast, when principals reported receiving technical assistance for 7 or fewer strategies, they reported that their school had fully implemented an average of only 5.7 strategies.”

Stein, L., LiCalsi, C., Kistner, A., Garcia-Piriz, D., & Melchior, K. (2017, April). Measuring impact and understanding implementation of School Improvement Grants in Massachusetts. Paper presented at the Annual Meeting of the American Educational Research Association, San Antonio, Texas. https://eric.ed.gov/?id=ED593310

From the ERIC abstract: “Despite largely mixed reports about the impact of School Improvement Grants (SIGs) on school improvement nationwide, Massachusetts has experienced proven success with these grants, termed School Redesign Grants (SRGs) in Massachusetts, as evidenced by comparative interrupted time series analyses, which show that SRGs have a significant impact on student achievement in both English language arts and mathematics one, two, and three years after grant receipt. This study builds upon previous impact studies of SRGs in Massachusetts, by incorporating additional schools and additional cohorts, and utilizes qualitative analyses to identify specific turnaround strategies or activities that distinguish schools able to improve student outcomes from schools struggling to do so, despite receiving the same grant and being afforded the same autonomies.”

Troppe, P., Milanowski, A., Garrison-Mogren, R., Webber, A., Gutmann, B., Reisner, E., et al. (2015). State, district, and school implementation of reforms promoted under the Recovery Act: 2009–10 through 2011–12 (NCEE 2015-4016). Washington, DC: National Center for Education Evaluation and Regional Assistance, Institute of Education Sciences, U.S. Department of Education. https://eric.ed.gov/?id=ED559930

From the ERIC abstract: “This report, based on surveys completed by all 50 State Education Agencies (SEAs) and the District of Columbia (DC) and nationally representative samples of districts and schools during spring 2011 and 2012, examines implementation of the key education reform strategies promoted by the Recovery Act in 2011-12, the extent to which implementation reflected progress since Recovery Act funds were first distributed, and challenges with implementation. Findings showed variation in the prevalence and progress of reform activities across the areas of reform assessed and by state, district, or school level. Implementation progress was most consistent across the areas of reform at the state level. At all levels, implementation challenges related to educator evaluation and compensation were common.”

Webber, A., Troppe, P., Milanowski, A., Gutmann, B., Reisner, E., & Goertz, M. (2014). State implementation of reforms promoted under the Recovery Act (NCEE 2014-4011). Washington, DC: National Center for Education Evaluation and Regional Assistance, Institute of Education Sciences, U.S. Department of Education. https://eric.ed.gov/?id=ED544746

From the ERIC abstract: “The American Recovery and Reinvestment Act (ARRA or the Recovery Act) of 2009 provided an unprecedented level of funding for K-12 education. The program created a ‘historic opportunity to save hundreds of thousands of jobs, support states and school districts, and advance reforms and improvements that will create long-lasting results for our students and our nation.’ Specifically, the Recovery Act allocated $70.6 billion in funding for K-12 education, of which $6.8 billion was awarded to states through a combination of newly created and existing grant programs, including the State Fiscal Stabilization Fund (SFSF) formula grants, Race to the Top (RTT) discretionary grants, and additional funding for the School Improvement Grant (SIG) program. In return for Recovery Act grants, recipients were required to commit to four specific core reforms or assurances: (1) Adopting rigorous college-ready and career-ready standards and high-quality assessments; (2) Establishing data systems and using data to improve performance; (3) Increasing teacher effectiveness and the equitable distribution of effective teachers; and (4) Turning around the lowest performing schools. By linking a commitment to the four assurances with receipt of funding, the Recovery Act signaled federal priorities; provided states, districts, and schools with incentives to initiate or intensify reforms in each of these areas; and encouraged states to pursue a combination of mutually supporting reform strategies. This report is part of the multi-year U.S. Department of Education (ED) evaluation ‘Charting the Progress of Education Reform: An Evaluation of the Recovery Act’s Role.’ ED seeks to understand through this evaluation how states, districts, and schools are working to implement the education reforms promoted by the Recovery Act. The current report focuses on whether, and how, state education agencies (SEAs) were implementing the reforms that the Recovery Act emphasized one year after the act was passed, and sets the stage for examination of implementation at the local level. It provides a picture of the prevalence and progress of the reform agenda promoted by the Recovery Act. A primary focus is SEA implementation of reforms in 2010-11, the first full school year after all Recovery Act funds were awarded. The education policies embedded in the Recovery Act were introduced into an ongoing stream of federal and state reform activity and states had undertaken some reforms before the act's passage. Therefore, the report also examines SEA implementation of reforms in 2009-10, and explores the extent to which 2010-11 reform activities represented progress. Specifically, this report addresses the following questions: (1) To what extent did SEAs report implementing key reform strategies promoted by the Recovery Act in the 2010-11 school year? (2) How much of the 2010-11 school year implementation reflects progress since the Recovery Act? and (3) What were the greatest reform implementation challenges for SEAs in the 2010-11 school year?”

Zimmer, R., Henry, G. T., & Kho, A. (2017). The effects of school turnaround in Tennessee’s Achievement School District and Innovation Zones. Educational Evaluation and Policy Analysis, 39(4), 670–696. https://eric.ed.gov/?id=EJ1158183

From the ERIC abstract: “In recent years, the federal government has invested billions of dollars to reform chronically low-performing schools. To fulfill their federal Race to the Top grant agreement, Tennessee implemented three turnaround strategies that adhered to the federal restart and transformation models: (a) placed schools under the auspices of the Achievement School District (ASD), which directly managed them; (b) placed schools under the ASD, which arranged for management by a charter management organization; and (c) placed schools under the management of a district Innovation Zone (iZone) with additional resources and autonomy. We examine the effects of each strategy and find that iZone schools, which were separately managed by three districts, substantially improved student achievement. In schools under the auspices of the ASD, student achievement did not improve or worsen. This suggests that it is possible to improve schools without removing them from the governance of a school district.”

Additional Organizations to Consult

Center on Reinventing Public Education (CRPE) – https://www.crpe.org/

From the website: “The Center on Reinventing Public Education (CRPE) is one of the nation’s leading sources for transformative, evidence-based ideas to improve education. To ensure all students are prepared for a rapidly changing future, we put forward rigorous research and policy analysis to help educators, policymakers, civic and community leaders, parents, and students themselves reimagine education systems and structures. Learn more about our current research.
We are a nonpartisan research center open to all possible solutions to measurably improve outcomes for all students. We work in the creative center across ideological lines to achieve strong, equitable schooling at scale, empower families, and inform and encourage effective governance.”
REL Southwest note: A CRPE publication, Measures of Last Resort: Assessing Strategies for State-Initiated Turnarounds, available at https://eric.ed.gov/?id=ED570486, provides descriptive data and a literature review of state-implemented takeover or turnaround initiates of schools and districts.

Center for School Turnaround and Improvement (CSTI) – https://centeronschoolturnaround.org/

From the website: “The Center for School Turnaround and Improvement (CSTI) at WestEd is a nationally recognized leader in the research and development of solutions that support systemic improvement for all schools. We work with you at all levels—from SEAs to districts to individual schools—to identify and help sustain evidence-based, promising practices that ensure equity and drive systemwide change for rapid improvement.”
REL Southwest note: CSTI is a federally funded comprehensive center. CSTI offers publications and tools to support school improvement, including District Readiness to Support School Turnaround: A Guide for State Education Agencies and Districts (2nd ed.), which is available at https://eric.ed.gov/?id=ED593091.

UChicago Consortium on School Research (UChicago Consortium) – https://consortium.uchicago.edu/about

From the website: “The UChicago Consortium conducts research of high technical quality that can inform and assess policy and practice in the Chicago Public Schools (CPS). We seek to expand communication among researchers, policymakers, and practitioners as we support the search for solutions to the problems of school reform. The UChicago Consortium encourages the use of research in policy action and improvement of practice, but does not argue for particular policies or programs. Rather, we help to build capacity for school reform by identifying what matters for student success and school improvement, creating critical indicators to chart progress, and conducting theory- driven evaluation to identify how programs and policies are working.”
REL Southwest note: UChicago Consortium offers a report, School Closings in Chicago: Staff and Student Experiences and Academic Outcomes, available at https://eric.ed.gov/?id=ED589712.

Methods

Keywords and Search Strings

The following keywords and search strings were used to search the reference databases and other sources:

  • [“school reform” AND (transformation OR turnaround OR closure OR restart OR charter)]
  • [(“school improvement” OR “school reform”) AND (transformation OR turnaround OR closure OR restart OR charter)]
  • [(“school improvement” AND “school reform”) AND (transformation OR turnaround OR closure OR restart OR charter)]

Databases and Resources

We searched ERIC for relevant, peer-reviewed research references. ERIC is a free online library of more than 1.7 million citations of education research sponsored by the Institute of Education Sciences (IES). Additionally, we searched the What Works Clearinghouse.

Reference Search and Selection Criteria

When we were searching and reviewing resources, we considered the following criteria:

  • Date of the publication: References and resources published from 2004 to present, were include in the search and review.
  • Search priorities of reference sources: Search priority is given to study reports, briefs, and other documents that are published and/or reviewed by IES and other federal or federally funded organizations, academic databases, including ERIC, EBSCO databases, JSTOR database, PsychInfo, PsychArticle, and Google Scholar.
  • Methodology: The following methodological priorities/considerations were given in the review and selection of the references: (a) study types—randomized control trials, quasi-experiments, surveys, descriptive data analyses, literature reviews, policy briefs, and so forth, generally in this order; (b) target population, samples (representativeness of the target population, sample size, volunteered or randomly selected, and so forth), study duration, and so forth; and (c) limitations, generalizability of the findings and conclusions, and so forth.
This memorandum is one in a series of quick-turnaround responses to specific questions posed by stakeholders in the Southwest Region (Arkansas, Louisiana, New Mexico, Oklahoma, and Texas), which is served by the Regional Educational Laboratory (REL) Southwest at AIR. This memorandum was prepared by REL Southwest under a contract with the U.S. Department of Education’s Institute of Education Sciences (IES), Contract ED-IES-91990018C0002, administered by AIR. Its content does not necessarily reflect the views or policies of IES or the U.S. Department of Education nor does mention of trade names, commercial products, or organizations imply endorsement by the U.S. Government.