Co-Principal Investigator: Thomas Ahn

Purpose: This study examines whether various forms of sanctions included in school accountability policies, including the No Child Left Behind Act and North Carolina's ABC program, improve student achievement with particular attention to traditionally underperforming disadvantaged students.

Project Activities: The main project activity will be secondary data analyses of North Carolina longitudinal student-level data. Using these data, the project will estimate the effects of a set of accountability sanctions on student achievement. Further, the study will examine the differential effects of accountability sanctions on different types of schools and on different types of students.

Products: The project will publish the results of the secondary data analysis regarding the effect of school sanctions on student achievement in peer reviewed journals.

Structured Abstract

Setting: This study will analyze data on the population of public elementary, middle, and high schools (including charter schools) in the state of North Carolina from 1994–95 to 2006–07.

Population: The population of North Carolina students taking standardized tests, generating disciplinary records, or enrolling in secondary courses in public schools statewide will be studied. The main foci will be on test results for students in grades 3–8, behavioral problems of fifth through tenth grade students, and the course-taking patterns of students in the early high school years.

Intervention: North Carolina schools are subject to both NCLB and North Carolina's accountability system. Accountability sanctions are based on school-level average test score performance and intended to encourage schools and teachers to adopt more effective ways of achieving predetermined educational goals. Sanctions may include both supportive and punitive actions. This project focuses on parental school choice, provision of supplemental school services, corrective action, and teacher bonuses (the last occurs under the North Carolina system).

Research Design and Analytic Strategy: The NCLB and the North Carolina accountability systems were implemented in ways that permit regression discontinuity analyses, which compare schools barely subjected to a sanction to schools that barely avoided it. Six different comparisons will be attempted: Six different comparisons will be attempted: 1) schools immediately above or below the AYP threshold; 2) schools with different AYP histories; 3) changes in sanctions that occurred in 7 districts in 2006; 4) schools above or below subgroup size thresholds; 5) schools just above or below the teacher bonus threshold; and 6) changes in the North Carolina accountability system.

Control Condition: Schools in the control condition will be those not directly exposed to or threatened with a sanction in a given year, either because they barely missed qualification or because a policy had not yet been implemented.

Key Measures and Outcomes: The key explanatory measures will be indicators for the set of sanctions implemented or threatened for a given year in a given school. In some cases, the previous year's sanction will substitute. Student outcome measures will be drawn from administrative records and include scores on state exams in reading and math in grades 3–8, scores on state end-of-course exams in a set of high school-level courses (including algebra I, geometry, algebra II, chemistry, biology, economic, legal, and political systems, and English I), adherence to a college preparatory track (based on course taking in math and science), and disciplinary problems. Teacher outcome measures will include teacher turnover, qualifications, and discretionary absences.

Data Analytic Strategy: The data will be analyzed using ordinary least squares analysis using a Huber-White correction for clustering. For binary outcomes (e.g., completing algebra I by the end of 9th grade), estimation will use a probit or logit model and for count outcomes (e.g., counts of disciplinary incidents) a poisson or negative binomial model will be used.

Products and Publications

Nongovernment report, issue brief, or practice guide

Ahn, T., & Vigdor, J. (2014). The Impact of No Child Left Behind's Accountability Sanctions on School Performance: Regression Discontinuity Evidence From North Carolina (No. w20511). National Bureau of Economic Research.

Ahn, T., & Vigdor, J. L. (2014). When Incentives Matter Too Much: Explaining Significant Responses to Irrelevant Information (No. w20321). National Bureau of Economic Research.

** This project was submitted to and funded under Education Policy, Finance, and Systems in FY 2009.