|Title:||An Evaluation of Turning Around North Carolina's Lowest Performing Schools: A State Partnership Evaluation|
|Principal Investigator:||Henry, Gary||Awardee:||Vanderbilt University|
|Program:||Evaluation of State and Local Education Programs and Policies [Program Details]|
|Award Period:||5 years (8/1/2015-7/31/2020)||Award Amount:||$4,944,014|
Co-Principal Investigator: Martez Hill, North Carolina State Board of Education and Department of Public Instruction
Partner Institutions: North Carolina State Board of Education and Department of Public Instruction; Education Policy Initiative at Carolina; RAND
Purpose:The research team will investigate the implementation and impact of a new turnaround plan for low-performing schools in North Carolina. The new plan, called the North Carolina Transformation (NCT) program, will target areas of high need for each school that will have quick pay-offs and implement a multi-tiered support system in each school that targets improved academic and behavioral outcomes. In addition, the NCT will align teacher professional development with data-informed needs of individual schools and guide principals of NCT schools in applying practices presented in statewide professional development.
Project Activities: Researchers will examine implementation of the NCT for 3 years and analyze impact of the NCT program over 4 years. The research team will analyze administrative data to determine the effects of NCT on student outcomes. They will use teacher and principal surveys and qualitative data collected from site visits to determine the quality and degree of fidelity of implementation of the transformation process in NCT schools. Researchers will also explore which underlying processes appear to mediate or suppress overall effects and unique effects on the lowest achieving students. Finally, the research team will implement a formal process for partners to communicate and exchange timely feedback about program implementation and effectiveness.
Products: The research team will provide evidence of the effect of the NCT program and publications in peer-reviewed journals and state briefings and presentations. Researchers will also increase the capacity of the North Carolina State Board of Education to conduct research.
Setting: Participating public schools are located across the state of North Carolina. North Carolina operated two prior rounds of turnaround programs and has been mandated to turnaround a new group of low performing schools.
Sample: The population consists of students, teachers, and leaders in schools that do not meet the state's growth standards. The 44 lowest performing schools based on the three-year average proficiency rate will be included in NCT and serve as the treatment sample for the evaluation.
Intervention: In 2015–16, North Carolina will begin implementing a new turnaround approach, the NCT program, that will (1) target areas of high need for each school that will have quick pay-offs; (2) implement a multi-tiered support system in each school that targets improved academic and behavioral outcomes; (3) align teacher professional development with data-informed needs of individual schools; and (4) guide principals of NCT schools in applying practices presented in statewide professional development.
Research Design and Methods: The research team will use two quantitative designs in this study: (1) researchers will estimate the effects on intermediate and student outcomes using a sharp regression discontinuity design that relies on the assignment of NC's schools not meeting growth to the NCT on the basis of a three year average proficiency rate; (2) researchers will use an interrupted time series design with a non-equivalent comparison group (difference-in-differences) to test moderated effects for the lowest achieving students and examine heterogeneity of effects. In addition, researchers will explore potential mediators using multi-level structural equation models. The research team will evaluate implementation fidelity by collecting surveys from teachers and principals at both treatment and comparison schools and will assess implementation quality using coaches' documentation of activities, on-site observations and interviews at the NCT schools.
Control Condition: In this study, low growth schools that missed the cut-off for NCT will serve as the comparison group (N=88).
Key Measures: Outcome measures for this study include student achievement (end of grade test scores), proficiency, graduation rate, and grade retention. Intermediate outcome measures include pedagogical effectiveness, engagement, and teacher retention. Implementation measures include teacher and principal reports about NCT processes, coaching intensity, and implementation fidelity checklists.
Data Analytic Strategy: For the outcome analysis, parametric and non-parametric local average treatment effect estimates will follow accepted practice for sharp regression discontinuity designs. The research team will examine validity of estimates by applying the What Works Clearinghouse standards. Researchers will implement difference-in-differences and school fixed effects estimators along with structural equation models to examine moderation and explore mediation.
*This grant was originally funded under the FY 2015 Evaluation of State Education Programs and Policies competition.