Impact Evaluation of Teacher and Leader Performance Evaluation Systems
Contract Information
Current Status:
This study has been completed.
Duration:
September 2011 – December 2017
Cost:
$21,523,477
Contract Number:
ED-IES-11-C-0066
Contractor(s):
American Institutes for Research
Instructional Research Group
University of Virginia
Teachstone
Discovery Education Association
Teachscape
The Danielson Group
Contact:
Educator performance evaluation systems are a potential tool for improving student achievement through increasing the effectiveness of the teacher and principal workforce. The Every Student Succeeds Act of 2015 allows the use of Title II Part A funds for the development and support of evaluation systems. This study was designed to examine the implementation of a package of performance evaluation system components and the impact of their use for formative purposes. These are components that states and districts might include in their evaluation systems. The evaluation system components in this study included measures of student achievement growth, classroom observations, and a measure of principal leadership. Based on these measures, teachers, leaders, and districts received timely and constructive feedback on teacher and principal performance. The components were implemented by the study's implementation team in a subset of schools in a sample of districts that did not already have an evaluation system similar to that being studied.
- What was the impact of the performance evaluation system on teachers' classroom practices, principal leadership, and student achievement?
- What were districts' and educators' experiences with implementation?
Eight districts participated in the study. Within each district, approximately 15 schools were randomly assigned to receive the study's intervention during 2012-2013 and 2013-2014, or to participate only in the district's usual performance evaluation system during the same time period. In intervention schools, each year, teachers of reading/English language arts and mathematics in grades 4 through 8 received four rounds of classroom observations and feedback sessions, based on one of two observation rubrics (the Framework for Teaching or the CLASS). District preference determined which rubric was used. These teachers also received information about their students' achievement growth in math and/or reading using a statistical model known as value-added. Principals in intervention schools received feedback on their performance based on the VAL-ED, which was administered twice per study year. The VAL-ED is an assessment of principals' leadership practices based on survey responses from the principal, their teachers, and their district supervisor.
The study used online vendor systems (e.g., Teachscape's system for tracking observations conducted using the Framework for Teaching rubric and producing performance reports for those observations) to look at the extent to which district implementation of the measures was as intended and whether ratings from the measures reliably distinguished performance. Based on principal and teacher survey data collected in the spring of each of the two study years, the study examined the extent to which educators' experiences and perceptions regarding performance evaluation differed between treatment and control schools. Impacts on teacher practices were measured through classroom observations conducted by the study team in the spring of the second year of implementation. Impacts on principal leadership were measured through survey reports of teachers in treatment and control schools in the spring of each of the two implementation years. Finally, impacts on students' achievement were estimated based on test score data from district records collected for both study years.
The final report for the study, titled The Impact of Providing Performance Feedback to Teachers and Principals, was released in December 2017.
Other publications from this study are listed below.
Reports on Implementation
- Early Implementation Findings From a Study of Teacher and Principal Performance Measurement and Feedback (November 2016)
Briefs
- Promoting Educator Effectiveness: The Effects of Two Key Strategies A brief that summarizes two strategies to promote educator effectiveness: the impact of using performance information for feedback to teachers and principals from this study and the impact of performance pay from another IES study of the Teacher Incentive Fund (March 2018)
A restricted-use file containing de-identified data is available for the purposes of replicating study findings and secondary analysis.
- The study's performance measures were implemented generally as planned. For instance, in both study years, teachers and principals received multiple rounds of ratings and feedback on their practices.
- The study's measures provided some information to identify educators who needed support, but provided limited information to indicate the areas of practice educators most needed to improve. For instance, classroom observation overall scores averaged over the year showed reliable differences across teachers. Sub-scores on particular dimensions of performance, however, had limited reliability to identify particular areas to guide teacher improvement.
- As intended, educators in treatment schools received more frequent feedback with ratings than did those in control schools.
- Feedback from the study's measures had some positive impacts on teachers' classroom practice, principal leadership, and student achievement. The intervention had a positive impact on teachers' classroom practice on one of two observation measures. It also had a positive impact on the two measures of principal leadership examined. In Year 1, the intervention had a positive impact on students' achievement in mathematics, amounting to about four weeks of learning. In Year 2, the impact on mathematics achievement was similar in magnitude but not statistically significant. The intervention did not have an impact on reading/English language arts achievement in either study year.