Executive Summary

This is the second and final report of the Middle School Mathematics Professional Development Impact Study, which examines the impact of providing a professional development (PD) program in rational number topics to seventh-grade mathematics teachers. An interim report (Garet et al. 2010) described the findings after one year of PD. The current report documents the impact after providing a second year of PD in a subset of the original participating districts and includes supplemental analyses that use data from both years of the study.

To improve teachers' knowledge and skill, federal policymakers have committed significant resources to teacher PD. In 2004–2005, for example, states and districts spent $1.5 billion in federal funds on teacher PD (Birman et al. 2007). There has, however, been only limited research evidence regarding the impact of PD on teacher and student outcomes.

Over the past decade, hundreds of studies have addressed the topic of teacher learning and PD (for reviews, see Borko 2004; Clewell, Campbell, and Perlman 2004; Kennedy 1998; Richardson and Placier 2001; Supovitz 2001; Yoon et al. 2007). However, the most recent review identified only 9 out of 1,343 studies of PD that had the types of rigorous designs—randomized control trials (RCTs) or quasi-experimental designs (QEDs)—that allow causal inferences to be made about the effectiveness of the PD strategies they examined. Four of those studies addressed the effect of teacher PD on mathematics achievement, but none focused on middle school mathematics (Yoon et al. 2007).

The U.S. Department of Education's National Center for Educational Evaluation and Regional Assistance (NCEE)—within the Institute of Education Sciences (IES)—initiated the Middle School Mathematics PD Impact Study to learn more about the role of PD in improving teacher effectiveness. Specifically, the study examines the impact of two years of a PD program for seventh-grade mathematics teachers that focuses on teachers' knowledge of rational number topics, including specialized mathematics knowledge that may be useful for teaching these topics. Rational numbers—fractions, decimals, percent, ratio, and proportion—are interrelated topics that are challenging for many seventh-grade students and are considered an essential foundation for algebra (National Mathematics Advisory Panel 2008).

The study also tests the effect of a PD program when implemented with a relatively large sample, in varied settings, and using multiple facilitators. The PD was delivered to approximately 100 treatment teachers in 12 districts in the first year of the study and approximately 50 treatment teachers in 6 districts in the second year. Ten facilitators from two separate PD organizations were involved over the course of the study. By contrast, the 9 studies with rigorous designs identified by Yoon and colleagues (2007) involved smaller samples of 5 to 44 teachers, and the PD programs were delivered by the individuals who developed them.

The second year of the study was designed to address two questions:

• What cumulative impact did providing two years of the specified PD program have on teacher knowledge of rational number topics?
• What cumulative impact did providing two years of the specified PD program have on student achievement in rational number topics?

The study produced the following core second-year results:

• The study's PD program was implemented as intended, but teacher turnover limited the average dosage received. On average, the treatment teachers in the second-year impact sample received 68 percent of the full intended dosage. Because some teachers left the study schools and others entered as the study progressed, not all teachers had the opportunity to experience the full dose of PD. (In particular, 22 of the 45 treatment teachers present at the end of the two-year PD program were not present at its beginning.) Relative to the hours of PD that each teacher could possibly have attended (that is, relative to the hours of PD that occurred after the teacher entered a study school), the teachers in the second-year impact sample averaged 89 percent of the possible dosage.
• At the end of the second year of implementation, the PD program did not have a statistically significant impact on teacher knowledge. There were no significant impacts on teachers' total score on a specially constructed teacher knowledge test (effect size = 0.05, p-value = 0.79) or on either of the test's two subscores. On average, 75.7 percent of the teachers in the treatment group correctly answered test items that were of average difficulty for the test instrument, compared with 74.7 percent of the teachers in the control group.
• At the end of the second year of implementation, the PD program did not have a statistically significant impact on average student achievement in rational numbers. There were no significant impacts on students' total score on a customized rational numbers test (effect size = -0.01, p-value = 0.94) or on either of the test's two subscores.