Co-Principal Investigators: Dee, Thomas; Thompson, Scott

Partnership Institutions: District of Columbia Public Schools; University of Virginia; Stanford University

Purpose: The partnership of the District of Columbia Public Schools (DCPS), University of Virginia (UVA), and Stanford University focused on the development and analysis of student, teacher, and school administrative data from DCPS to understand the effects of IMPACT: The DCPS Effectiveness Assessment System for School-Based Personnel on student achievement and guide its ongoing design refinements. In addition, researchers worked closely with policymakers to understand better the strategic and institutional aspects of IMPACT to insure sufficiently nuanced research designs and interpretations.

Project Activities: This partnership designed a set of initial research questions, developed an analytic database to address those questions, and conducted descriptive and causal analysis on the effect of IMPACT. Partners met every two weeks to discuss the progress of ongoing research and data collection, adjust research based on policy considerations. The partnership established an advisory board that intended to provide feedback to the research team.

Key Outcomes:

• On average, when a teacher left DCPS and that teacher was replaced by a new teacher, students of the teacher performed about eight percent of a standard deviation better in math and no differently in reading (Adnot et al., 2017).

Structured Abstract

Setting: This project took place in District Columbia Public Schools.

Sample: The analytic sample included all teachers and students in DCPS from 2009–10 (IMPACT's first year) through 2016–17.

Education Issue: In AY 2009–10, DCPS implemented IMPACT: The DCPS Effectiveness Assessment System for School-Based Personnel. IMPACT is a high-stakes teacher evaluation system intended to improve teaching and student achievement. IMPACT was intended to set clear expectations, provide systematic, informed feedback on teaching performance, facilitate collaboration, inform teacher training, support, and retain/reward effective instructional personnel. During IMPACT's first 3 years, DCPS increased the pay of teachers rated as "highly effective," while it terminated the employment of teachers rated as "ineffective."

Research Design and Methods: This project team used descriptive analysis and quasi-experimental design.

Key Measures: Key measures included teacher mobility (e.g., leaving their school or the district to teach at somewhere else) teacher characteristics, and student achievement in math and reading.

Data Analytic Strategy: The strategies included descriptive analyses and regression discontinuity analyses to examine the influence of IMPACT on (1) the differences across time in teacher applicant characteristics, (2) teacher mobility within DCPS and between DCPS and DC charters, and (3) aspects of teaching that were most malleable in the context of strong incentives. The partnership team also explored whether findings differ by teacher groups (i.e., instructors of tested grades/subjects versus the remainder who represent the majority of the DCPS teacher population), by different weights applied to evaluation system component scores, by use of the PARCC assessments versus the prior assessments, by teacher uptake of instructional coaching, and by student/community characteristics (e.g., parent's education, household income, community safety). Additionally, researchers investigated the effect of IMPACT on student achievement in math and reading.

Products and Publications

ERIC Citations: Find available citations in ERIC for this award here.

Selected Publications:

Adnot, M., Dee, T., Katz, V., & Wyckoff, J. (2016). Teacher turnover, teacher quality and student achievement in DCPS (Working Paper 153). National Center for Analysis of Longitudinal Data in Education Research (CALDER).

Adnot, M., Dee, T., Katz, V., & Wyckoff, J. (2017). Teacher turnover, teacher quality, and student achievement in DCPS. Educational Evaluation and Policy Analysis, 39(1), 54–76.

Dee, T. S., James, J., & Wyckoff, J. (2019). Is effective teacher evaluation sustainable? Evidence from DCPS (CEPA Working Paper No. 19–09). Stanford Center for Education Policy Analysis.

Dee, T. S., James, J., & Wyckoff, J. (2021). Is effective teacher evaluation sustainable? Evidence from District of Columbia Public Schools. Education Finance and Policy, 16(2), 313–346.

James, J., & Wyckoff, J. H. (2020). Teacher evaluation and teacher turnover in equilibrium: Evidence from DC public schools. AERA Open, 6(2), 2332858420932235.

Phipps, A. R., & Wiseman, E. A. (2021). Enacting the rubric: Teacher improvements in windows of high-stakes observation. Education Finance and Policy, 16(2), 283–312.