Co-Principal Investigators: Nathaniel Schwartz and Laura Booker (Tennessee Department of Education), Eric Taylor (Harvard University), Ellen Goldring and Jason A. Grissom (Vanderbilt University

Partner Institution: Tennessee Department of Education (TDOE)

Purpose: Building on existing research collaborations between the Tennessee Department of Education (TDOE) and researchers at both Vanderbilt and Brown/Harvard Universities, researchers for this project will examine the implementation and impact of the Evaluation Partnership Program (EPP) on teacher and student outcomes. The EPP is a central component of the state’s teacher evaluation system and its ongoing efforts to use the system to promote instructional improvement. Researchers will also focus on developing the research partnership to build on TDOE’s capacity to use research to advance agency priorities and to conduct rigorous program evaluation and manage randomized interventions. In addition, the grant will build capacity among all partners to develop and sustain a research agenda that can provide rigorous and actionable evidence to inform state policy.

Project Activities: This partnership will analyze the effectiveness and statewide implementation of the Evaluation Partnership Program (EPP), an intervention that pairs teachers with low and high scores on an assessment of teaching practices that is a component of the State’s teacher evaluation system, in order to help improve teaching. TDOE will implement the intervention. Researchers will explore moderators and mechanisms, the reasons that principals and teachers choose to participate (or not) in the program, and the types of actions the state can take to promote participation in EPP. Partners will also calculate the cost of implementing EPP.

Products: The research team will provide evidence of the impact of the EPP intervention and develop publications in peer-reviewed journals as well as state briefings and presentations. The partnership will also establish a long-term research agenda and increase the capacity of the TDOE to conduct research about key topics of interest to State policymakers and educators.

Structured Abstract

Setting: This study will take place in all school levels (elementary, middle, and high school) located across the State of Tennessee.

Sample: Study participants will include approximately 1400 schools (80% of all Tennessee schools), 41,000 teachers, and 340,000 students.

Intervention: The Evaluation Partnership Program (EPP) partners a teacher with low scores in particular areas of instructional practice (e.g., “Questioning” or “Lesson Structure and Pacing”) with a teacher with high scores in those same areas. The Tennessee Department of Education (TDOE) uses teacher evaluation data to identify possible collaborative teaching partners in each school based on a matching algorithm. The participating principals are provided with a list of possible dyads. Principals provide feedback and make the final determination of EPP dyads to work together in their school. Principals are encouraged to meet with each teacher to determine if the teacher wishes to participate, to arrange a kick-off meeting including the dyad and principal, and to meet with dyads to discuss their goals and teacher partnership activities. Dyads are encouraged to work together throughout the year. Intervention materials consist of an intervention guide, letter with instructions/suggestions to participating principals, and letter with instructions/suggestions to participating teachers.

Research Design and Methods: Researchers will utilize a randomized controlled trial to evaluate the effectiveness of this intervention. Researchers will also explore moderators and mechanisms, measured from statewide survey data (e.g., principal support for the intervention, pre-EPP teacher evaluation score school type, proportion of teachers participating in EPP within each school, number of skills matched). In addition, the research team will also explore the reasons that principals and teachers choose to participate (or not) in the program and the types of actions the state can take to promote participation in EPP.

Control Condition: Control schools will not have access to EPP or any alternative intervention specific to the control condition during the first two years of the project. In Year 3 of the project, control schools will receive EPP.

Key Measures: The primary outcomes in this study include student achievement as measured by state tests (i.e., Tennessee Comprehensive Assessment Program (TCAP) tests for students in grades 3 through 8 and end-of-course exams for high school students), teacher instructional practice as measured by the classroom observation component of the teacher evaluation system, and teacher views of the evaluation system as measured by the Tennessee Educator Survey (TES). Researchers will also use the TES, site visits, and interviews to explore questions about implementation.

Data Analytic Strategy: Researchers will use multivariate regression for intent-to-treat analysis, treatment-on-the-treated analysis, and analyses on the heterogeneity of impacts based on varying circumstances (e.g., principal support for the intervention, pre-EPP teacher evaluation score school type, proportion of teachers participating in EPP within each school, number of skills matched) after 1 year and after 2 years of participating in EPP. The research team will summarize implementation information gathered from the Tennessee Educator Survey, site visits, and case studies.

*This grant was originally funded under the FY 2015 Evaluation of State Education Programs and Policies competition.