|Title:||An Evaluation of Turning Around North Carolina's Lowest Performing Schools: A State Partnership Evaluation|
|Principal Investigator:||Henry, Gary||Awardee:||Vanderbilt University|
|Program:||Evaluation of State and Local Education Programs and Policies [Program Details]|
|Award Period:||5 years (8/1/2015-7/31/2020)||Award Amount:||$4,944,014|
Previous Award Number: R305E150017
Co-Principal Investigators: Rebecca Herman, RAND Corporation; Julie Marks, Education Policy Initiative at Carolina; Deanna Townsend-Smith, North Carolina State Board of Education and Department of Public Instruction
Purpose: The research team is investigating the implementation and impact of a school turnaround plan for low-performing schools in North Carolina. The plan, called the North Carolina Transformation (NCT) program, identified and targeted increasing each school's needs for building leadership and instructional capacities.
Project Activities: Researchers examined implementation of NCT and subsequent state services to support low-performing schools for three years and are analyzing the impact of NCT over three years. The research team is analyzing administrative data to determine the effects of NCT on student outcomes. They are using teacher and principal surveys and qualitative data collected from site visits to determine the quality and fidelity of implementation of the transformation process in NCT schools. Researchers are also exploring the underlying processes that appear to mediate or suppress overall effects and the effects on the lowest achieving students. Finally, the research team is engaged in a formal process for partners to communicate and exchange timely feedback about program implementation and effectiveness.
Products: The research team will provide evidence of the effect of the NCT program and publications in peer-reviewed journals and state briefings and presentations. Researchers will also increase the capacity of the North Carolina State Board of Education to conduct research.
Setting: Participating public schools are located across the state of North Carolina. North Carolina operated two prior rounds of turnaround programs and has been mandated to turnaround a new group of low-performing schools.
Sample: The population consists of students, teachers, and leaders in schools that do not meet the state's performance standards. Seventy-five of the state's lowest performing schools based on the 2015 proficiency rate received services under NCT and serve as the treatment sample for the evaluation.
Intervention: In 2015–16, North Carolina began implementing a new turnaround approach, NCT, to (1) target areas of high need for each school in order to produce quick performance improvements; (2) implement a multi-tiered support system in each school that targets improved academic and behavioral outcomes; (3) align teacher professional development with data-informed needs of individual schools; and (4) guide principals of NCT schools in applying practices presented in statewide professional development.
Research Design and Methods: The research team is using two quantitative designs in this study: (1) researchers are estimating the effects of NCT on intermediate and student outcomes using a regression discontinuity design that relies on the assignment of eligible schools not meeting growth to the NCT based on the 2014–-15 schoolwide proficiency rate; (2) researchers are using an interrupted time series design with a non-equivalent comparison group (difference-in-differences) to test moderated effects for the lowest achieving students and examine heterogeneity of effects. In addition, researchers are exploring potential mediators and suppressors in both the RD and DiD frameworks. The research team is evaluating implementation fidelity by collecting surveys from teachers and principals at both treatment and comparison schools and assessing implementation quality using coaches' documentation of activities, on-site observations and interviews at the NCT schools.
Control Condition: In this study, schools that missed the proficiency rate cut-off for NCT will serve as the comparison group (N=84).
Key Measures: Outcome measures for this study include student achievement (end of grade and end of course test scores), proficiency, graduation rate, grade retention, and chronic absenteeism. Intermediate outcome measures include principal and teacher turnover and working conditions. Implementation measures include teacher and principal reports about NCT processes, coaching intensity, and implementation fidelity.
Data Analytic Strategy: For the outcome analysis, parametric and non-parametric local average treatment effect estimates follow accepted practice for regression discontinuity designs. The research team is examining validity of estimates by applying the What Works Clearinghouse standards. Researchers are estimating difference-in-differences and school fixed effects estimators along to examine moderation and explore mediation.
*This grant was originally funded under the FY 2015 Evaluation of State Education Programs and Policies competition.