What can we learn from a district-managed school restart strategy in Texas?
REL Southwest and the Texas Education Agency (TEA) work together in our School Improvement Research Partnership to support low-performing schools in the state. This post provides a researcher’s perspective on a recently completed study, Effects of a District-Managed Restart Strategy for Low-Performing Schools in Texas.
Every year, the Texas Education Agency (TEA) provides grants via the School Action Fund to give school districts the opportunity to implement turnaround strategies at low-performing schools. In 2015/16, the Dallas Independent School District (ISD) implemented a district-managed restart strategy in seven of its lowest performing schools. By the end of 2019, a total of 29 schools across four urban and suburban districts in Texas had implemented the strategy.
This Texas turnaround strategy was holistic and comprehensive, involving five core components: (1) strategic staffing, (2) instructional excellence, (3) additional learning time, (4) social and emotional learning (SEL) supports, and (5) strong partnerships with families and community organizations. TEA partnered with REL Southwest to better understand the implementation and outcomes of the strategy. We spoke with Dr. Angelica Herrera, the principal investigator, to learn more about the origin of this study, its findings, and implications. Our thanks to Angelica for sharing her research perspective on this Texas restart study!
How did this study come about?
It evolved over time through the work of REL Southwest’s School Improvement Research Partnership and our main partners at the Texas Education Agency (TEA). The goal of the partnership is to enhance TEA’s ability to evaluate and continuously improve low-performing schools and districts by supporting promising and evidence-based practices that, when applied, significantly reduce the number of low-performing schools and increase the number of students attending high-performing schools. REL Southwest has supported TEA through training and assistance projects and applied research studies. When this study first came about, TEA was very interested in having us conduct an impact study on their school actions, which is the term they use for turnaround models.
Unfortunately, a lack of data due to the COVID-19 pandemic meant we would not have accountability data to support an impact study on all the school actions. We discussed with TEA whether any school action had historical data and learned that Dallas ISD was the first district to implement the district-managed restart strategy, beginning in 2015/16. (Read a related blog post by Dallas ISD’s Jolee Healey about the district’s restart model implementation.)
How did you measure the impact of the restart strategy?
We used a quasi-experimental design called a comparative interrupted time series that can be used for causal inference. This method compares the outcomes of an intervention group (in this case schools that experience a district-managed restart strategy) with outcomes for a comparison group of schools (in this case, schools with similar performance and student characteristics before the intervention) after the intervention occurs, to determine the effect of the intervention after controlling for prior trends. This method is appropriate for contexts in which an abrupt policy change occurs, such as a school restart, and in which preintervention and postintervention data are available. Dallas and other districts that implemented the strategy from 2015/16 to 2018/19 had two or more years of postintervention data available. We were able to use the comparative interrupted time series method because we could access 10 years of preintervention extant data from the Texas Education Research Center. We could examine the changes that happened in this group of schools compared to similar schools.
Although the original idea to study all the school actions that are funded by the School Action Fund wasn’t possible, TEA was enthusiastic about investigating one school action using this rigorous design. They were interested because the district-managed restart strategy is expanding across the state. So, it worked out for them and for us!
Key Findings
- Student achievement and attendance improved after schools implemented the restart strategy.
- Nearly all restart schools met accountability standards within the first three years of implementation.
- Nearly 80 percent of the teachers at schools in the year before implementation of the restart strategy left before the beginning of the restart school year.
- Educators who arrived at restart schools were more likely to have more than three years of experience and to have an advanced degree than those who left or stayed.
- Interviews with district and school leaders suggested that recruiting high-performing teachers to relocate to restart schools was time consuming and that the grant-funded salary stipend might not have been a large enough incentive.
What key takeaway or finding would you highlight from this study?
I think a major highlight is that the restart strategy had a pretty large and positive effect on students’ reading and math scores. For the most part, the schools that implemented it had been chronically low performing for years, receiving low ratings from the state accountability system. It was amazing to see the degree that students’ scores on state reading and math tests went up within the first year of implementation. Schools went from being an F-rated school to a C-rated school or even a B-rated school. They did it so quickly! In education, it can typically take years to see improvements in achievement.
Source: REL Southwest study, Effects of a District-Managed Restart Strategy for Low-Performing Schools in Texas, page 9.
Given the results from the study, what are the greater implications for Texas and beyond?
Implications of the study may help TEA in allocating resources to implement and support school actions. For example, the agency may consider expanding the district-managed restart model to additional low-performing schools in light of study findings that suggest positive effects on student academic performance during the first three years of implementation. Additionally, district leaders might consider policies, programs, and incentives for increasing the supply of high-performing teachers who can staff low-performing schools participating in the restart strategy. District leaders also might want to consider the implications of scaling up the restart strategy, with the goal of identifying potential unintended consequences on other schools in the district.
Several other state education agencies (SEAs) are similar to Texas when it comes to providing funding for districts, and there’s national interest to learn what the effects were of this restart strategy and how it impacted students. This is an exciting addition to the field of education policy reform, especially when we consider the evidence of positive impacts on reading and math scores.
What future research do you think can be built upon the work in this study?
I think that’s a really good question. In our report, we discuss the benefit of having a more in-depth look at implementation. For example, how did districts identify teachers for the restart schools, especially districts without a teacher evaluation system? How did districts address the other component of the turnaround strategy we found was challenging, family and community partnerships?
Then, thinking about the teachers, it would be interesting to look at how districts had to recruit and incentivize teachers to move to these schools. Even the $10,000 salary stipend was not enough to incentivize some teachers. How do you increase the pool of high-performing teachers?
Finally, leaders at TEA may consider collecting data and conducting additional research on districts’ implementation of the five components of the restart strategy. A more in-depth examination of implementation would provide information and lessons learned that districts could use, especially those that are considering implementing the strategy or that are in the early phases of implementation. In the future, TEA leaders could also consider examining additional years of data from restart schools to learn whether improvements in student achievement and attendance are sustained beyond the three years of grant funding for the staff stipends. Future research could also examine changes in principal or teacher performance using classroom observation scores from the Texas Teacher Evaluation and Support System or measures of principals’ or teachers’ contribution to student test score growth.
Ultimately, the positive findings and evidence from this study should motivate TEA and other agencies to continue to study these models.
For more information on school improvement and turnaround strategies:
- REL Southwest infographic: Lessons Learned From a Statewide District-Managed Restart Strategy for Low-Performing Schools in Texas
- REL Southwest blog: A Texas School Restart Experience: Q&A with Dallas ISD’s Jolee Healey
- REL Southwest infographic: Highlighting the Evidence Behind Strategic School Actions and School Improvement Strategies
- REL Southwest video training series: Understanding and Applying the Evidence Base for the Texas Education Agency’s Strategic School Actions
- REL Southwest training series resources: Using Logic Modeling for Statewide School Improvement Efforts Training
- REL Southwest infographic: Theory of Change for a Successful School Improvement Partnership
- Texas Education Agency: Center for School Actions
- REL Central report: State policies for intervening in chronically low-performing schools: A 50-state scan
- REL Central guide: A Guide to Identifying Similar Schools to Support School Improvement