Co-Principal Investigator: Randall Reback, Barnard College

Purpose: The purpose of this project is to study whether the incentives built into the No Child Left Behind Act (NCLB) have a net positive impact on students' academic achievement, students' non-academic outcomes, and school resource allocation. Under NCLB, states annually determine whether public schools satisfy Adequate Yearly Progress (AYP) based on the fraction of students demonstrating proficiency on a statewide exam. Public schools have strong incentives to satisfy AYP: failure triggers sanctions that escalate over consecutive years, and states are required to publish annual school report cards, which can affect school prestige, local property values, and financial rewards to schools and teachers. Using quasi-experimental methods, the project will evaluate the efficacy of the accountability pressure put on schools that expect to be near the margin for meeting AYP using a nationally representative sample of schools and students.

Project Activities: Drawing on data from Standard & Poors and from states' departments of education, the project will assemble a national dataset that includes school and student subgroup AYP outcomes in academic years 2002–03 through 2005–06, as well as states' AYP rules and regulations. Information on school demographics and Title I status will be included from the National Center for Education Statistics's Common Core of Data. In addition, the AYP data will be matched by school to panel data on a nationally representative sample of elementary school children from the Early Childhood Longitudinal Survey-Kindergarten Class of 1998–99, a panel data set on students who began kindergarten in the school year 1998–99. These students were surveyed again in school years 2001–02, 2003–04, and 2006–07, when most would have been in third, fifth, and eighth grade, respectively.

Products: This work will produce published reports of the secondary data analyses, which will contribute to the knowledge base on the effects of accountability policies on a range of outcomes, including student achievement, students' non-academic outcomes, and school resource allocation. An additional outcome will be a publicly available, national dataset including school and student subgroup AYP outcomes in the 2002–03 through 2005–06 academic years. This dataset will also include states' AYP rules and regulations.

Structured Abstract

Setting: The analytic sample includes a nationally representative sample of U.S. kindergarteners, based on the restricted-use version of the Early Childhood Longitudinal Survey–Kindergarten Class of 1998–99 (ECLS-K).

Population: The ECLS-K sample was designed to be nationally representative of kindergartners, their teachers, and schools. The ECLS-K follows the same children from kindergarten through the end of elementary school. Data collection took place in the fall and the spring of the school years 1998–99 and 1999–2000 (kindergarten and first grade), and in the spring of the school years 2001–02, 2003–04, and 2006–07 (third grade, fifth grade, and eighth grade). The ECLS-K data from the school year 2006–07 are not currently available, but should be released in 2009 and will therefore be included in the study. The restricted-use version of the ECLS-K identifies individual schools and can therefore be linked with school accountability measures.

Intervention: The intervention in this study is the accountability pressure placed on schools by NCLB. Schools on the margin of making AYP are subject to this pressure, and can be expected to alter their behavior in order to avoid the sanctions for failing to make AYP under NCLB.

Research Design and Methods: This project will use quasi-experimental methods to examine the effects of the accountability incentives that NCLB creates for public schools on student achievement, students' non-academic outcomes, school resource allocation, and the behavior of teachers within classrooms.

This project will improve upon existing research on NCLB by (a) being national in scope, as opposed to focusing on a single state or city (b) following individual students over time instead of comparing average student outcomes across schools, and (c) exploiting intra- and inter-state variation in schools' AYP performance to make stronger causal claims about the impact of NCLB on both student outcomes and school resources. The project will compare (a) differences in outcomes for schools on the AYP margin and not on the AYP margin in the same state with (b) differences in outcomes for similar schools in other states where neither school is on the margin. Since each state designates its own standardized test and proficiency rate cutoffs for AYP, there are numerous cases where schools have moderate chances of satisfying their own state's AYP requirements but would almost certainly fail or almost certainly pass AYP if they were located in other states. The use of panel data will allow for the control of students' initial achievement levels and characteristics to implement a "difference-in-differences" approach that will separate the effects of NCLB incentives from other factors influencing student outcomes.

Control Condition: The comparison condition in the study will be the absence of accountability pressure placed on schools that expect to be far from the AYP margin.

Key Measures: The ECLS-K contains survey responses concerning individual students' observed emotional and social behaviors, schools' academic and non-academic services, teachers' attitudes, teachers' time-use as well as student reading and math test scores.

Data Analytic Strategy: The identification strategy rests on the assumption that the accountability pressure of NCLB affects schools that expect to be near the margin of making AYP, but does not affect schools that expect to be far from this margin. In addition, the substantial variation across states in the difficulty of making AYP will allow the project to compare schools with very similar student characteristics and prior achievement but different accountability pressure. This motivates the "difference-in-differences" approach to estimating the causal impact of NCLB on student outcomes. The project will compare (a) differences in outcomes for students in the same state enrolled in schools that are and are not near the AYP margin to (b) differences in outcomes of students in other states enrolled in similar schools that are both not near the AYP margin. To isolate the effect of attending a school on the margin of making AYP, the project will control for students' prior achievement levels and characteristics and a host of other factors that might separately contribute to outcomes.

Products and Publications

Journal article, monograph, or newsletter

Davidson, E., Reback, R., Rockoff, J., and Schwartz, H.L. (2015). Fifty Ways to Leave a Child Behind: Idiosyncrasies and Discrepancies in States' Implementation of NCLB. Educational Researcher, 44 (6), 347–358.

** This project was submitted to and funded under Education Policy, Finance, and Systems in FY 2009.