Effects of the PD Interventions During the Implementation Year

Teachers’ Knowledge of Early Reading Content and Instruction

• Teachers who were assigned to the institute series only group (treatment A) or the institute series plus coaching group (treatment B) scored significantly higher on the overall teacher knowledge total score, in comparison with the control group teachers (effect sizes = 0.37 and 0.38, respectively; see figure E-1). In addition, treatment group A and B teachers scored significantly higher than control group teachers on the word-level subscale (effect sizes = 0.35 and 0.39, respectively). The estimated effects were not statistically significant for the meaning-level subscale (effect sizes = 0.21 for treatment A and 0.26 for treatment B), although they were positive.
• The institute series was designed to nurture teacher knowledge, whereas the coaching was designed to help teachers translate this knowledge into practice. Therefore, coaching was not expected to have an impact on teacher knowledge. The additional PD delivered through coaching (tested by comparing treatment B with treatment A) did not produce a statistically significant added effect on overall teacher knowledge or either of the teacher knowledge subscales (effect sizes for the difference in impacts between treatments B and A were 0.01 on the total score, 0.04 on the word-level subscale, and 0.05 on the meaning-level subscale).

Teachers’ Instructional Practice: Use of Explicit Instruction, Independent Student Activity (ISA), and Differentiated Instruction (DI) During Reading Instruction

• The treatment group A and B teachers used explicit instruction to a significantly greater extent than control group teachers (effect sizes = 0.33 and 0.53, respectively). See figure E-2.
• There were no statistically significant impacts on the use of the other two types of instructional practices focused on in the study (independent student activity and differentiated instruction), although a comparison of teachers in treatment group B and teachers in the control group showed an estimated effect size of 0.22 for the use of independent student activity.
• The differential impact of coaching on teacher practices was not statistically significant. The estimated effect size for the impact of the intervention on explicit instruction was 0.53 for teachers who participated in coaching along with the institute series (treatment B), and 0.33 for teachers who participated only in the institute series (treatment A), a difference of 0.21. Similarly, the estimated effect size for the impact on independent student activity was 0.22 for treatment group B teachers, and 0.05 for treatment group A teachers, a difference of 0.17. The estimated effect sizes for differentiated instruction, however, were negative for both treatment A and treatment B (-0.05 and -0.02, respectively) with a difference of 0.03. None of these differences between treatment A and treatment B were statistically significant.

Students’ Reading Achievement

• The improvement in teacher knowledge and the increased explicitness of teachers’ instruction caused by the PD intervention did not translate into improvements in student reading achievement as measured by standardized tests given by each district. Neither the institute series alone (treatment A) nor the combination of institutes, seminars, and coaching (treatment B) produced a statistically significant impact on the main outcome measure: standardized student reading test scores (effect sizes = 0.08 and 0.03, respectively; see figure E-3). Nor was there a statistically significant effect on the percent of students scoring at or above the overall baseline mean reading score (3.48 and -2.35 percent, respectively).

Effects of the PD Interventions During the Follow-Up Year

• The year after the PD was concluded, there was no statistically significant effect of either the institute series alone (treatment A) or the institute series plus coaching (treatment B) on teacher’s knowledge of reading content (figure E-4) or their use of the instructional practices encouraged by the study PD (figure E-5). With one exception (see below), the difference in teacher impacts between the implementation year and the follow-up year was not statistically significant; thus, we cannot conclude with confidence that any positive impacts during the implementation year declined over time.
• The estimated effect of treatment B on the use of explicit instruction was lower by a statistically significant margin in the fall of the follow-up year (-0.03) than in the implementation year (0.53; figure E-5).
• Neither treatment had statistically significant impacts on student achievement in the follow-up year (figures E-6 and E-7).
• There were no statistically significant differences between the follow-up and implementation year impacts for either the standardized student test score (figure E-6) or the dichotomous outcome (figure E-7).

Top

---

¹¹ It should be noted that the treatment B teachers reported an average of 71 hours of coaching rather than the 62 reported by the study coaches; however, the teacher survey item this estimate is based on did not limit teachers’ responses to only the study-provided coaching. Therefore, teacher estimates may also include coaching and mentoring from other sources.