Structured Abstract
Setting
Participants will be drawn from K-12 public schools located in medium to large United States public school districts and from the Washoe County School District in Reno, Nevada, which is the second largest in the state, operating 84 schools and serving over 58,000 students will participate.
Sample
The sample includes 84 principals and approximately 3,000 teachers and 3,000 students from Washoe County and approximately 1,000 district administrators from a national sample.
For the study, researchers use a standards-based performance evaluation system to evaluate the school leadership skills of principals. In particular, they will evaluate the relationship of principal performance ratings to measures of student achievement, teacher collective efficacy, teacher professional community, and academic excellence norms. Researchers will also perform a nation-wide survey on the uses of standard-based leadership evaluations in large and medium-size U.S. school districts.
Research design and methods
In phase 1 of the study, half of the 84 Washoe principals will be randomly assigned to be evaluated using a new standards-based evaluation system consisting of specific performance standards in eight domains and a set of rating scales describing four levels of performance in each. Researchers will compare treatment group responses to evaluation to those of the remaining principals. In phase 1, researchers will also partner with the Educational Research Service (ERS) to administer the national survey to administrators of medium and large school districts in the U.S. on principal evaluation and performance-based compensation for principals. ERS is a nonprofit research organization affiliated with the main national associations of school administrators and has conducted surveys of various issues of interest to school administrators in the past. In phase 2, researchers will evaluate all 84 Washoe principals under the new system, allowing researchers to study the relationships between principals' standards-based evaluation ratings, school culture, and academic achievement. Teacher and student surveys will also be administered in phase 2 of the study. All principals will be surveyed each year and researchers will interview a simple random sample of all principals each year to ascertain principal understandings of performance expectations, changes in behavior to these expectations, and perceptions of alignment between the performance standards and district strategies for improving student achievement. Lastly, in phase 2, researchers plan to use the national survey data to identify several sites for further study of performance evaluations and pay.
Control condition
In phase 1, half of the Washoe principals will serve as a control group for the 42 principals receiving the treatment. Participants in the control condition will be evaluated under the Washoe district's previous evaluation system.
Key measures
For the standards-based evaluation, principals will be evaluated on (a) goal setting and evaluation, (b) analysis and monitoring of student achievement, (c) instructional leadership, (d) communications, (e) community relations, (f) organizational management, (g) decision-making, and (h) professional development. Three constructs will be measured to represent school culture: professional community, collective efficacy, and academic press. Student achievement measures include reading/language arts and mathematics scores for students at all schools in grades 3 to 8. The researchers will develop measures for the national survey instrument that will cover the following content areas: methods of principal evaluations, alignment of principal evaluations with other human resource management programs, and performance-based compensation for principals.
Data analytic strategy
Two-level hierarchical linear models (HLMs) will be used to derive a value-added measure of student achievement in reading/language arts and mathematics. Simple correlations will be used to assess the strength of the relationships between principal performance ratings and each criterion separately for both student achievement and the mediating variables (professional community, collective efficacy, and academic press). Next, HLMs will be used to estimate the effects of principal evaluation on the mediating variables and student achievement. Lastly, researchers will test for mediation by comparing the relationship of (a) performance ratings to student achievement, (b) performance ratings to the mediating variables, and (c) the mediating variables to student achievement, controlling for evaluations scores. For the national sample, descriptive analyses will be used to evaluate interview and survey responses.
People and institutions involved
IES program contact(s)
Products and publications
Publications:
ERIC Citations: Find available citations in ERIC for this award here.
Select Publications:
Journal articles
Kimball, S.M., and Milanowski, A.T., and McKinney, S.A. (2009). Assessing the Promise of Standards-Based Performance Evaluation for Principals: Results From a Randomized Trial. Leadership and Policy in Schools, 8(3): 233-236.
Kimball, S.M., Heneman, H.G. III, and Milanowski, A. (2007). Performance Evaluation and Compensation for Public School Principals: Results From a National Survey. ERS Spectrum, 25(4): 11-21.
Questions about this project?
To answer additional questions about this project or provide feedback, please contact the program officer.
