Co-Principal Investigators: Ellen Goldring and Michael Knapp (University of Washington)

Purpose: This study will refine and expand the use of an assessment tool that helps school principals to observe classroom instruction and provide feedback and support to teachers. Specifically, it will refine and validate the Diagnostic Assessment of Instructional Leadership Capacity (DAILC), an online, video-based assessment tool that aims to measure school leaders' capacity to:

• observe and notice the quality of classroom instruction (noticing)
• interpret their observations and provide productive feedback to the teacher (interpreting), and
• use the observations to plan professional development for the teaching staff (using).

The DAILC is a refinement and expansion of the 5D Assessment, a widely used tool developed by the Center for Educational Leadership (CEL) in which participants view a video of classroom teaching and then provide written responses. These responses are scored by CEL raters and reported to school districts.

Project Activities: Activities fall into two main phases: transforming the 5D Assessment into the new DAILC (Phase 1) and validating the enhanced instrument for different uses (Phase 2). In Phase 1, researchers will revise the existing scoring rubric, modify the written response instructions, and redesign the assessment to use multiple short videos (rather than one longer video). In Phase 2, researchers will test whether the new instrument distinguishes between more or less effective instructional leaders, predicts student achievement growth, and predicts potential hires' future success as instructional leaders.

Products: This measurement project will produce a fully developed and validated performance based assessment tool designed to measure school leaders' instructional leadership capacity. The research team will also produce peer reviewed publications.

Structured Abstract

Setting: The research activities will take place in Washington and Tennessee.

Sample: The study uses different samples for the two phases. In order to refine the measure, the team will use data from previous users of the 5D Assessment and will gather additional data from participants recruited from the University of Washington's Danforth Principal Preparation Program Network. The validation activities will use participants from the Tennessee Academy of School Leaders (leaders in their first 2 years in an administrative position within Tennessee public schools).

Assessment: The DAILC is an online, video based assessment tool to measure school leaders' instructional leadership capacity. Participants will view one or more videos of literacy and/or math instruction and respond to three questions in writing: What do you notice about teaching and learning in this classroom? What conversation would you want to have with this teacher? How, if at all, does this inform your thinking about and planning for professional development? Written responses are scored by trained raters and given to the district for formative or diagnostic purposes.

Research Design and Methods: In Phase 1 (Refinement and Enhancement), researchers will conduct four sub-studies using samples from Washington state, to systematically explore possible refinements and enhancement of the existing 5 Assessment, resulting in the new, fully-developed DAILC. These sub-studies include: reducing measurement redundancy in the existing scoring rubric (sub-study 1), elaborating and assessing the content validity of the scoring rubric (sub-study 2), investigating the value of more scaffolded response options (sub-study 3), and investigating the comparability, reliability, and generalizability of responses to multiple, shorter videos (sub-study 4). In Phase 2 (Validation), researchers will conduct three sub-studies to validate the DAILC. They will test whether the DAILC distinguishes between more or less effective instructional leaders (sub-study 5), use student growth-modeling to assess the extent to which DAILC scores predict student growth (sub-study 6), and investigate the value of using DAIC scores at the hiring stage to predict candidates' future success as instructional leaders (sub-study 7).

Control Condition: There is no control condition in this study.

Key Measures: The DAILC measures three instructional leadership constructs: (1) observing and noticing the quality of classroom instruction, (2) interpreting observations for teacher feedback, and (3) using observational data to plan and develop supports for teacher improvement. For the Tennessee-based validation sub-studies, other measures include the Tennessee Education Acceleration Model Administrator Evaluation Rubric; student achievement measures from the Tennessee Comprehensive Assessment Program and end-of-course exams; and measures of teacher perceptions of school leadership.

Data Analytic Strategy: Multiple techniques, including confirmatory factor analysis, generalized partial credit analyses, and classical test theory, will be used to help refine the scoring rubric. Generalizability study and decision study approaches will be used to determine the optimal numbers of videos to use in the assessment. Multiple techniques will also be used to validate the measure for different uses, including correlational analyses, confirmatory factor analyses, linear regression, multi-level modeling, discriminant analyses, and student growth modeling.

*This grant was originally funded through the Improving Education Systems topic in the 2015 Education Research Grants (84.305A) program.