Study Findings After Two Years of Treatment

Implementation Findings

• Across the six districts that participated in the study for two years, the average number of hours of institutes, seminars, and coaching delivered was 118 hours, which was 4 hours more than the intended dosage of 114 hours. During the institutes and seminars, the PD providers delivered an average of 93 percent of the intended hours of professional development in each year of the study. With regard to coaching, the treatment group teachers received an average of 97 percent of the intended hours in the first year and 132 percent of the intended hours in the second year.
• The treatment group teachers attended an average of 77 hours of study PD and reported participating in 63.6 hours more mathematics-related PD than the control group teachers. The average hours of study PD attended represented 68 percent of the intended dose of 114 hours and 66 percent of the total 118 PD hours implemented across the two years. However, relative to the hours of PD that each teacher could possibly have attended (that is, relative to the hours of PD that occurred after the teacher entered a study school), the teachers in the second-year impact sample averaged 89 percent of the possible dosage.
• Teacher turnover limited the maximum possible PD dosage and the magnitude of the treatment-control group service contrast. Twenty-two of the 45 treatment teachers teaching regular seventh-grade mathematics classes at the end of the two-year PD program were not present at its beginning. Most turnover occurred over the summer between the two years of implementation.¹¹

Impact Findings

Impact on Teachers' Knowledge of Rational Number Topics and How to Teach Rational Number Topics

• At the end of the second year of implementation, the PD program did not have a statistically significant impact on overall teacher knowledge. On average, 75.7 percent of teachers in the treatment group correctly answered test items of average difficulty for the test instrument, compared with 74.7 percent for teachers in the control group (effect size = 0.05, p-value = 0.79). (See Figure ES-1.)
• The PD program did not have a statistically significant impact on either of the teacher knowledge subscale scores. On average, 79.9 percent of treatment group teachers correctly answered CK test items of average difficulty for the test instrument, compared with 84.1 percent of control group teachers (effect size = -0.21, p-value = 0.25). On average, 65.8 percent of treatment group teachers correctly answered SK test items of average difficulty for the test instrument, compared with 56.2 percent of control group teachers (effect size = 0.36, p-value = 0.09). (See Figure ES-1.)

View Figure ES-1. Impact of the PD Program on Teacher Knowledge at the End of the Second Year

Impact on Student Achievement in Rational Numbers

• At the end of the second year of implementation, the PD program did not have a statistically significant impact on average student achievement as measured by the NWEA Rational Number Test Total Score. Students in treatment schools on average scored 219.90 scale score points, compared with 219.97 scale score points for the control group (effect size = -0.01, p-value = 0.94). (See Figure ES-2.)
• The PD program did not have a statistically significant impact on either of the student achievement subscale scores. On the Fractions and Decimals Score, students in treatment schools on average scored 218.15 scale score points, compared with 218.36 scale score points for students in control schools (effect size = -0.01; p-value = 0.84). On the Ratio and Proportion Score, students in treatment schools on average scored 221.71 scale score points, compared with 221.57 scale score points for students in control schools (effect size = 0.01; p-value = 0.89). (See Figure ES-2.)

View Figure ES-2. Impact of the PD Program on Student Mathematics Achievement at the End of the Second Year