Previous Award Number: R305A110242
Previous Awardee: The Urban Institute

Co-Principal Investigators: Michael Hansen

Purpose: The project investigates two ways in which schools and teachers might behave strategically when facing pressure from school accountability systems based on student test scores. First, schools might provide more attention to students who have full academic year eligibility. Under the No Child Left Behind Act of 2011 (NCLB), a school is only accountable for a student's performance if that student has attended that school for a certain number of days and states are required to define that time, which is known as a full academic year (FAY). Schools might give greater attention to students with FAY eligibility than for otherwise similar students who do not have such eligibility because the latter's test scores will not affect the evaluation of the school. Second, teachers might teach to the test (i.e. focus on subjects for which test scores are used in the school accountability system). The project will analyze student-level administrative data from Florida to examine these two possible behaviors.

Project Activities: Researchers will use student-level administrative data from Florida to determine whether there are important pattern differences that might reflect school and teacher behaviors in response to accountability pressure. Florida uses an October cutoff date to determine which students have FAY eligibility. The project will use the cutoff date to implement a regression discontinuity design for a comparison of grade 3 through grade 5 students who entered the school at which they took the test right before the October eligibility cutoff to those who entered right after the cutoff. Should schools prioritize their resources on eligible students, a large discontinuity in student achievement on standardized tests would be expected. This would be especially true in those schools facing the strongest accountability pressure (measured using Florida's program of grading schools using a scale of A to F). To explore teaching to the test, the project will correlate test scores of students in grades 3 to 10 from a high stakes state test used for accountability and a low stakes state test in schools facing different levels of accountability pressure. Classrooms with different levels of expected ability will be studied.

Products: Products include articles published in peer reviewed journals describing the results of the analyses.

Structured Abstract

Setting: The data used in this study come from the state of Florida's individual-student and school-level record system.

Population: Regarding schools responses to FAY eligibility, the analyses will include all Florida elementary students in grades three through five for the school years 2002/03 to 2005/06. Regarding teaching to the test, the analyses will include all Florida students in grades 3 through 10 for the school years 2002/03 through 2007/08.

Intervention: The Florida school accountability system employs performance-based rewards, sanctions, and assistance in order to achieve proficiency benchmarks. Schools are graded on an A–F scale. The grading system was revised in the 2001/02 school year to include the Florida Curriculum Assessment Test (FCAT-SSS) reading and math scores for grades 3 through 10 along with year-to-year student progress in these subjects with special attention to reading gains of students in the lowest reading quartile at each school. The researchers hypothesize that the Florida school accountability systems, developed partly in response to NCLB, encourages strategic behaviors by schools and teachers that may lead to unintended differential treatment of students and subject matter.

Research Design and Methods: The first part of this project uses a regression-discontinuity design to estimate how FAY eligibility affects student achievement by comparing the high-stakes test performances of students in grades three through five (for the years 2002/03 through 2005/06) who entered the school at which they took the test right before the October eligibility cutoff to those who entered right after the cutoff and thus became ineligible. Second, using a longitudinal administrative dataset of students in grades 3 through 10 in Florida spanning the 2002/2003 to 2007/2008 schools years and a variety of statistical models, researchers will compare student performance across a high-stakes and a low-stakes test to look for evidence of teaching to the test. The theoretical model predicts better performance on tests that count and by FAY-eligible students as well as larger impacts in schools that face higher accountability pressures and have classrooms where average student ability falls close to the proficiency cutoff.

Control Condition: The comparisons to be made are between FAY-eligible and FAY-ineligible students and between student scores on a high stakes test and a low stakes test. A further breakdown of these comparisons is by the level of accountability pressure a school is under.

Key Measures: The outcome measures include student test scores in reading and math on a high stakes test (FCAT-SSS) and a low stakes test (FCAT-NRT). School accountability pressure will be measured using Florida's grading of each school on an A–F scale with schools receiving a D or F under the greatest pressure. The administrative data will provide students' date of entry into the school they were tested in. This data will be used to identify students who arrived a month before or a month after the October cutoff date for FAY eligibility and these students will examined in the regression discontinuity design.

Data Analytic Strategy: Researchers will use a modified difference-in-difference strategy, quantile regression methods, and regression discontinuity methods in this study. Researchers will also investigate differences across subgroups of schools and students.

** This project was submitted to and funded under Education Policy, Finance, and Systems in FY 2011.