Co-Principal Investigator: Francisco Martorell

Purpose: A growing number of states and school districts have established policies that link grade promotion decisions to standardized test performance. The assumption underlying such policies is that an additional year of instruction in the same grade will help struggling students acquire the skills needed to succeed in future grades. Grade retention remains highly controversial and considerable uncertainty exists regarding the causal effect of retention on student outcomes. The researchers will examine the impact of grade retention under a comprehensive student promotion policy instituted by New York City Department of Education (NYCDOE). Prior work by the researchers has shown that retention in 5th grade under this policy had significant positive effects on English language arts and mathematics achievement in 6th and 7th grade. This study will extend the earlier work by examining the effects of retention in grades 3 through 8 on high school persistence, behavioral outcomes, and academic measures.

Project Activities: Using administrative student panel data from NYCDOE the researchers will compare the longer-term (high school) success of two groups of students who were at-risk of being retained in their grade because of their scores on the spring district standardized tests. They will compare the high school outcomes of those who did not score high enough on a summer test to be promoted to those scored high enough on the summer test to be promoted. The main difference between the intervention and comparison groups will be retention status.

Products: The products of this project will be evidence of the efficacy of NYCDOE's retention policy for grades 3 through 8. The evidence will be directly provided to the NYCDOE through the participation of department personnel on the project and to the research community and other education practitioners and policymakers through peer reviewed publications.

Structured Abstract

Setting: All non-charter New York City Department of Education schools serving general education students.

Sample: Students in grades 3 through 8 subject to the promotion policy in the 2003–04 through 2011–12 academic years and followed through the 2015–2016 school year.

Intervention: The NYCDOE promotion policy ties grade retention decisions to standardized test performance. Students in grades 3 through 8 scoring in the lowest performance category (Level 1) on either the math or English Language Arts (ELA) spring assessments are at risk for being retained while those who score in the next highest category (Level 2) on both subjects are eligible for promotion. The policy provides students multiple attempts to demonstrate eligibility for promotion. Students with Level 1 scores on the mathematics or ELA spring assessment can demonstrate Level 2 proficiency through a portfolio review in June. Those who do not demonstrate Level 2 proficiency attend the City's summer instructional program, the Summer Success Academy (SSA). At the SSA's conclusion, students take a city summer assessment in the subjects in which they scored at Level 1 on the spring assessment. Those not demonstrating Level 2 performance on the summer assessment have the opportunity to do so through an August portfolio review. Those not demonstrating Level 2 or better performance by the end of this process (and not receiving an exemption by their principal or superintendent) are retained in grade.

Research Design and Methods: The researchers will use a regression discontinuity research design that exploits the fact that grade retention is largely determined by whether a student scores below the Level 2 cutoff on the summer assessment. Students scoring just above or just below the Level 2 cutoff are likely to be similar in all other dimensions so that comparisons between students who score just above and below the Level 1 summer assessment cutoff can be used to identify the effect of grade retention. As not all students who score below the Level 2 cutoff on the summer assessment are retained (some are promoted based on the August portfolio review or through exemptions), the researchers will use a fuzzy regression discontinuity design that uses the variation in the likelihood of being retained at the summer assessment Level 2 cutoff as an instrumental variable for retention. Provided that determinants of student outcomes other than the probability of retention do not change at the Level 2 cutoff, this approach will identify the impact of retention for students near the Level 2 cutoff whose retention status is affected by scoring above or below the cutoff. An implementation study will document the status of students who scored Level 1 in the summer (retained, promoted through portfolio review, or promoted through special exemption) and the distribution of these statuses across the 32 primary geographic school districts and by subgroups of interest.

Control Condition: Students in the comparison group are those who are promoted to the next grade because of scoring Level 2 on the summer assessment who are nearest the threshold between Levels 1 and 2. The key difference between these two groups is that students in one are retained while those in the other are promoted.

Key Measures: The impact analysis will address measures of high school persistence (e.g., dropout, high school graduation, credits accumulated) and behavioral problems (unexcused absences and disciplinary infractions). Exploratory analyses will address available high school achievement measures, including the New York Regents exams.

Data Analytic Strategy: Instrumental variables estimates of the effect of retention will be obtained using 2-stage least squares regression. Models will be estimated separately by grade in case effects vary by the grade of retention. The role of other moderating variables (e.g., race, gender, prior retention status) as well as mediating variables (e.g., future placement into special education, retention in subsequent grades) will be examined to obtain more detail on how retention affects student outcomes.