Project Activities
This partnership designed a set of initial research questions, developed an analytic database to address those questions, and conducted descriptive and causal analysis on the effect of IMPACT. Partners met every two weeks to discuss the progress of ongoing research and data collection, adjust research based on policy considerations. The partnership established an advisory board that intended to provide feedback to the research team.
Structured Abstract
Setting
This project took place in District Columbia Public Schools.
Sample
The analytic sample included all teachers and students in DCPS from 2009-10 (IMPACT's first year) through 2016-17.
In AY 2009–10, DCPS implemented IMPACT: The DCPS Effectiveness Assessment System for School-Based Personnel. IMPACT is a high-stakes teacher evaluation system intended to improve teaching and student achievement. IMPACT was intended to set clear expectations, provide systematic, informed feedback on teaching performance, facilitate collaboration, inform teacher training, support, and retain/reward effective instructional personnel. During IMPACT's first 3 years, DCPS increased the pay of teachers rated as "highly effective," while it terminated the employment of teachers rated as "ineffective."
Initial research
This project team used descriptive analysis and quasi-experimental design.
Key measures
Key measures included teacher mobility (e.g., leaving their school or the district to teach at somewhere else) teacher characteristics, and student achievement in math and reading.
Data analytic strategy
The strategies included descriptive analyses and regression discontinuity analyses to examine the influence of IMPACT on (1) the differences across time in teacher applicant characteristics, (2) teacher mobility within DCPS and between DCPS and DC charters, and (3) aspects of teaching that were most malleable in the context of strong incentives. The partnership team also explored whether findings differ by teacher groups (i.e., instructors of tested grades/subjects versus the remainder who represent the majority of the DCPS teacher population), by different weights applied to evaluation system component scores, by use of the PARCC assessments versus the prior assessments, by teacher uptake of instructional coaching, and by student/community characteristics (e.g., parent's education, household income, community safety). Additionally, researchers investigated the effect of IMPACT on student achievement in math and reading.
Key outcomes
- On average, when a teacher left DCPS and that teacher was replaced by a new teacher, students of the teacher performed about eight percent of a standard deviation better in math and no differently in reading (Adnot et al., 2017).
People and institutions involved
IES program contact(s)
Project contributors
Partner institutions
District of Columbia Public Schools
Stanford University
Products and publications
Publications:
ERIC Citations: Find available citations in ERIC for this award here.
Selected Publications:
Adnot, M., Dee, T., Katz, V., & Wyckoff, J. (2016). Teacher turnover, teacher quality and student achievement in DCPS (Working Paper 153). National Center for Analysis of Longitudinal Data in Education Research (CALDER).
Adnot, M., Dee, T., Katz, V., & Wyckoff, J. (2017). Teacher turnover, teacher quality, and student achievement in DCPS. Educational Evaluation and Policy Analysis, 39(1), 54-76.
Dee, T. S., James, J., & Wyckoff, J. (2019). Is effective teacher evaluation sustainable? Evidence from DCPS (CEPA Working Paper No. 19-09). Stanford Center for Education Policy Analysis.
Dee, T. S., James, J., & Wyckoff, J. (2021). Is effective teacher evaluation sustainable? Evidence from District of Columbia Public Schools. Education Finance and Policy, 16(2), 313-346.
James, J., & Wyckoff, J. H. (2020). Teacher evaluation and teacher turnover in equilibrium: Evidence from DC public schools. AERA Open, 6(2), 2332858420932235.
Phipps, A. R., & Wiseman, E. A. (2021). Enacting the rubric: Teacher improvements in windows of high-stakes observation. Education Finance and Policy, 16(2), 283-312.
Supplemental information
Co-Principal Investigators: Dee, Thomas; Thompson, Scott
Questions about this project?
To answer additional questions about this project or provide feedback, please contact the program officer.
