Skip Navigation
Print Evaluations

Elementary School Math Professional Development Impact Evaluation

Contract Information

Current Status:

This study has been completed.

Duration:

September 2012 – March 2017

Cost:

$8,157,019

Contract Number:

ED-IES-12-C-0080

Contractor(s):

American Institutes for Research
Harvard University
Measured Decisions
Clowder Consulting

Contact:

Title II, Part A of the Elementary and Secondary Education Act provides monetary resources to improve teacher quality. One way to affect teacher quality, and thus teacher effectiveness, is by providing in-service teacher training, also called professional development (PD). Currently there is relatively little rigorous empirical evidence on the components of an effective PD program (i.e., the importance of emphasizing content knowledge and how that knowledge might be delivered to teachers and then infused into the classroom). At the same time, there has been renewed policy interest in fostering a globally-competitive STEM workforce, which will require teachers to effectively impart mathematics to their students. Of particular concern are upper elementary school teachers (e.g., grades 4 and 5), who are typically less likely to have strong mathematical training and yet are asked to teach math topics that are foundational for more advanced courses in middle school and beyond.

This evaluation focused on estimating the impact of an intensive math PD intervention focused on enhancing teacher content knowledge and integrating this knowledge into the classroom. The intervention had three components: Intel Math, Mathematics Learning Communities (MLC), and Video Feedback Cycles (VFC). Intel Math, the core of the intervention, is a widely used, 80-hour professional development course focused on developing teachers' understanding of K–8 mathematics. The MLC was aligned to the Intel Math program and offered teachers opportunities to connect their learning to the classroom through a professional learning community that focused on student thinking in a series of 5 two-hour sessions. The VFC provided further support for connecting their learning to the classroom by allowing teachers to practice what they learned three different times using actual lessons that were videotaped and reviewed by analysts with expertise in assessing the mathematical quality of instruction. For each round of videos, analysts shared the results with a district-based math coach (who also led the MLCs), and the coach provided 1-on-1 feedback to the teacher, linking the classroom practice to the relevant MLC and Intel Math content.

  • What was the average impact on (a) teachers' content knowledge, (b) teachers' classroom practices, and (c) student achievement of offering a specialized professional development intervention (in this case, Intel Math, Mathematics Learning Communities, and Video Feedback Cycles) relative to "business-as-usual" professional development?
  • How was the professional development intervention implemented?

Approximately 200 volunteer fourth-grade math teachers from 6 districts around the country participated in this randomized controlled trial. Within each participating school, fourth-grade teachers were randomly assigned to receive either the PD intervention or business-as-usual PD.

The Intel Math component consisted of 80 instructional hours during the summer, covering K-8 math topics aligned with the common core state standards. About 90 percent of the course was focused on content, while 10 percent was focused on pedagogy. The MLC component consisted of 10 instructional hours split into 5 interactive sessions throughout the fall, and involved activities such as analyzing student work to help teachers integrate content learned from Intel Math into the classroom. The VFC component consisted of 3 instructional hours split into 3 1-on-1 sessions throughout the fall, and involved practicing the delivery of three different lessons and receiving content-based feedback on the quality of instruction using expert coders and math coaches to help integrate content learned from Intel Math and MLCs into the classroom.

Tests of math content knowledge were administered to all study teachers in the summer (as baseline), fall (after Intel but prior to MLCs and VFCs), and spring (after the entire intervention had been delivered). Extant state math assessment data for these teachers' students were collected from districts for the pre and post study periods. In addition, study teachers were observed using the Mathematical Quality of Instruction (MQI) classroom observation protocol once in the fall (after Intel but prior to MLCs and VFCs) and twice in the spring (after the entire intervention had been delivered). These data collection activities helped inform the study's first research question. Additional teacher survey data and data collected by the implementation team helped inform the second question.

A report, titled Focusing on Mathematical Knowledge: The Impact of Content-Intensive Teacher Professional Development, along with a video recap and study snapshot of findings, was released in September 2016.

Other publications from this study are listed below.

Briefs

Blog Posts

A restricted-use file containing de-identified data is available for the purposes of replicating study findings and secondary analysis.

On the Math PD Program:

  • The PD had a positive impact on teacher knowledge. Average scores on a study-administered math test were 21 percentile points higher for teachers who received the study PD than for those who did not.
  • The PD had a positive impact on some aspects of instruction. Average ratings of teachers' use and quality of math explanations in the classroom were 23 percentile points higher for teachers who received the study PD than for those who did not.
  • The PD did not have a positive impact on student achievement. Students of teachers who received the PD scored, on average, 2 percentile points lower than control teachers' students on both a study-administered math test and state assessment. In general, this difference was not statistically significant.

On Three Content-Focused PD Programs Tested Across Three IES Evaluations:

  • The PD improved teachers' knowledge and some aspects of their practice.
  • Improving teachers' knowledge and practice did not lead to positive impacts on student achievement.
  • Most of the measured aspects of teachers' knowledge and practice were not correlated with student achievement.
  • The consistent pattern of findings suggests that future studies might seek to better understand on what aspects of teacher knowledge and practice PD should focus, and how PD can achieve a larger impact on knowledge and practice that also impacts student achievement.