Measuring principals' effectiveness: Results from New Jersey's first year of statewide principal evaluation

States and districts across the country are implementing new principal evaluation systems that include measures of the quality of principals' school leadership practices and measures of student achievement growth. Because these evaluation systems will be used for high-stakes decisions, it is important that the component measures of the evaluation systems fairly and accurately differentiate between effective and ineffective principals. This requires the measures to be reliable (consistent across raters and observations) and valid (accurately measuring true principal performance). This study examined data from 2013/14, the first year of statewide implementation. It examined four statistical properties of the system's component measures: the variation in overall and component measure ratings across principals, the year-to-year stability of overall and component measure ratings, the correlations between component measure ratings and characteristics of students in the schools, and the correlations among component measure ratings. Information about these properties of the measures can inform efforts to improve the principal evaluation system and revise the guidance districts receive. Key findings include: (1) Nearly all principals received effective or highly effective overall ratings; (2) The percentage of principals who received highly effective overall ratings was lower for principals who were evaluated on school median student growth percentiles than for principals who were not evaluated on this measure; (3) Principal practice instrument ratings and school median student growth percentiles had moderate to high year-to-year stability; (4) Several component measure ratings--school median student growth percentile ratings, teachers' student growth objective ratings, and principal practice instrument ratings--as well as the overall rating, had low, negative correlations with student socioeconomic disadvantage; and (5) Principals' ratings on component measures had low to moderate positive correlations with each other. The following are appended: (1) Description of districts participating in the 2012/13 pilot; (2) Design of principal evaluation system; (3) Data used in the study; (4) Variation in ratings on the component measures; (5) Changes in the principal practice instrument and school median student growth percentiles and their associated ratings across years; (6) Correlations of component measure ratings with student background. characteristics for assistant principals; and (7) Correlations among component measure ratings for assistant principals.