Study of Innovative School Leadership Performance Evaluation Systems
Principals are expected to set and maintain the conditions necessary for student success and are increasingly held responsible for the improvement of student achievement. The same accountability demands require that school districts evaluate school leadership as well as assist principals in obtaining the skills needed to improve student performance. However, little rigorous research exists on specific school leadership skills that are associated with student success, assessment practices that can identify leaders who possess these skills, and procedures that might assist principals in developing these skills. The purpose of this project is to provide evidence of the potential efficacy of standard-based leadership evaluations in helping principals develop the proficiencies needed to support improved student achievement.
Purpose: Increased accountability demands on school principals to improve student achievement have led to a need for research on how school leadership skills are associated with student success, on assessment practices that can identify leaders who possess these skills, and on how to help principals develop these skills. The purpose of this project is to provide evidence of the potential efficacy of standard-based leadership evaluations in helping principals develop the proficiencies needed to support improved student achievement.
Setting: Participants will be drawn from k-12 public schools located in medium to large United States public school districts and from the Washoe County School District in Reno, Nevada, which is the second largest in the state, operating 84 schools and serving over 58,000 students will participate.
Populations: 84 principals and approximately 3,000 teachers and 3,000 students from Washoe County and approximately 1,000 district administrators from a national sample.
Assessment: For the Washoe case study, researchers will evaluate the relationship of principal performance ratings to measures of student achievement, teacher collective efficacy, teacher professional community, and academic excellence norms. Researchers will also perform a nation-wide survey on the uses of standard-based leadership evaluations in large and medium-size U.S. school districts.
Research Design and Methods: In Year 1 of the study, half of the 84 Washoe principals will be randomly assigned to be evaluated using a new standards-based evaluation system consisting of specific performance standards in eight domains and a set of rating scales describing four levels of performance in each. Researchers will compare treatment group responses to evaluation to those of the remaining principals. In Year 1, researchers will also partner with the Educational Research Service (ERS) to administer the national survey to administrators of medium and large school districts in the U.S. on principal evaluation and performance-based compensation for principals. ERS is a nonprofit research organization affiliated with the main national associations of school administrators and has conducted surveys of various issues of interest to school administrators in the past. In Year 2, researchers will evaluate all 84 Washoe principals under the new system, allowing researchers to study the relationships between principals' standards-based evaluation ratings, school culture, and academic achievement. Teacher and student surveys will also be administered in the 2nd year of the study. All principals will be surveyed each year and researchers will interview a simple random sample of all principals each year to ascertain principal understandings of performance expectations, changes in behavior to these expectations, and perceptions of alignment between the performance standards and district strategies for improving student achievement. Lastly, in Year 2, researchers plan to use the national survey data to identify several sites for further study of performance evaluations and pay.
Control Condition: In Year 1, half of the Washoe principals will serve as a control group for the 42 principals receiving the treatment. Participants in the control condition will be evaluated under the Washoe district's previous evaluation system.
Key Measures: For the standards-based evaluation, principals will be evaluated on a) goal setting and evaluation; b) analysis and monitoring of student achievement; c) instructional leadership; d) communications; e) community relations; f) organizational management; g) decision-making; and h) professional development. Three constructs will be measured to represent school culture: professional community, collective efficacy, and academic press. Student achievement measures include reading/language arts and mathematics scores for students at all schools in grades 3-8. The researchers will develop measures for the national survey instrument that will cover the following content areas: methods of principal evaluations, alignment of principal evaluations with other human resource management programs, and performance-based compensation for principals.
Data Analytic Strategy: Two-level hierarchical linear models (HLMs) will be used to derive a value-added measure of student achievement in reading/language arts and mathematics. Simple correlations will be used to assess the strength of the relationships between principal performance ratings and each criterion separately for both student achievement and the mediating variables (professional community, collective efficacy, and academic press). Next, HLMs will be used to estimate the effects of principal evaluation on the mediating variables and student achievement. Lastly, researchers will test for mediation by comparing the relationship of a) performance ratings to student achievement, b) performance ratings to the mediating variables, and c) the mediating variables to student achievement, controlling for evaluations scores. For the national sample, descriptive analyses will be used to evaluate interview and survey responses.
Journal article, monograph, or newsletter
Kimball, S.M., and Milanowski, A.T., and McKinney, S.A. (2009). Assessing the Promise of Standards-Based Performance Evaluation for Principals: Results From a Randomized Trial. Leadership and Policy in Schools, 8(3): 233–236.
Kimball, S.M., Heneman, H.G. III, and Milanowski, A. (2007). Performance Evaluation and Compensation for Public School Principals: Results From a National Survey. ERS Spectrum, 25(4): 11–21.