|Title:||Investigating How and Under What Conditions Effective Professional Development Increases Student Achievement in Elementary Science|
|Principal Investigator:||Little, Judith||Awardee:||University of California, Berkeley|
|Program:||Effective Instruction [Program Details]|
|Award Period:||2 years (9/1/2015-8/31/2017)||Award Amount:||$699,354|
Co-Principal Investigator: Joan Heller (Heller Research Associates) and Luke Weisman Miratrix (Harvard University)
Purpose: The purpose of this project is to explore and analyze data collected from an evaluation of three professional development interventions in science. There is wide variation in the ways professional development programs improve teachers' experience and skills, making it difficult to determine which features of professional development interventions are associated with better teacher or student outcomes. To obtain a better understanding of the contributions of teacher professional development programs, the researchers will analyze data to identify aspects of professional development interventions that are associated with positive teacher and student outcomes in 4th grade science.
Project Activities: The researchers will conduct secondary data analysis to generate hypotheses about why and how professional development improves teacher's science content knowledge, pedagogical content knowledge, and classroom practices in order to advance student science outcomes.
Products: The products of this project include a better understanding of how different features of professional development programs are associated with improvements in teacher and student outcomes to help guide the design of future professional development programs. Researchers will also produce peer-reviewed publications.
Setting: This study relies on data collected from schools in Arizona, California, Washington, Massachusetts, North Carolina, and Alabama during 2007 to 2008.
Sample: Data were collected from 8 school sites, with the final sample including 283 teachers and their 7,000 students. Across the schools, teacher characteristics ranged from 80%–90% female, 60–70% White, 9–14% Black, 1–10% Hispanic or Latino, and 3–4% Asian. The teachers in the sample had a wide range of teaching and professional development experience, spanning from novice to veteran teachers.
Intervention: The researchers will analyze data from a randomized control trial that evaluated three in-service professional development interventions focusing on the teaching of electric circuits in Grade 4 science. Teachers were randomly assigned to one of three professional development interventions – Teaching Cases, Looking at Student Work, or Metacognitive Analyses – or to a control condition. Intervention teachers were expected to teach all of their classroom electric circuits lessons after they completed the professional development course. The design included two rounds of implementation and data collection during the study year, and additional data collection in a follow-up year to assess delayed treatment effects. Results from the evaluation showed that all three in-service treatment interventions increased both teacher and student science content knowledge when compared to business-as-usual, with results maintained a year later.
Research Design and Methods: For the current project, the researchers will analyze qualitative data focusing on the implementation of the professional development interventions, and examine associations between teachers' classroom practices and student learning. Researchers will analyze samples of professional development course videos from 24 courses with up to eight three-hour sessions per course, and classroom videos of two consecutive lessons from a subsample of 33 teachers. Analyses of the professional development implementation will be mapped against the theory of change. The qualitative analysis will focus on coding the professional development facilitators' and classroom teachers' focus on conceptual learning goals, accuracy, breadth, and depth of science content in small and whole-group discussions of evidence, and support for the development of learners' science content knowledge, using professional development rating scales and classroom observation protocols. In addition, the researchers will conduct quantitative analyses using multi-level modeling to explore the variations within and across professional development interventions. Specifically, the primary mediators that researchers will explore include teachers' classroom practices that support conceptual learning goals for students, consideration of student thinking, and student engagement in generating explanations of science ideas and observations. Student outcomes include science content knowledge and quality of written explanations of text answers.
Control Condition: Teachers in the control condition will receive business-as-usual professional development in science.
Key Measures: Data sources for the study include observations and videotapes of professional development course implementation and classroom implementation, facilitator surveys and interviews, and teacher background surveys. Student and teacher tests of science content knowledge were also measured using a researcher developed assessment.
Data Analytic Strategy: Researchers will use StudioCode software, a software program for processing video data, to code and annotate video data. Boolean search functions can be applied within single timelines/videos or across multiple timelines/videos, yielding quantitative matrix displays and summary descriptive statistics. Researchers will explore relationships between professional development and classroom practices using permutation-based methods to provide a robust minimal-assumption analysis. The research team will examine relationships between professional development and student test outcomes using multi-level modeling with teachers and professional development conditions as random effects. Researchers will model teacher outcomes as a two-level model with teachers nested within schools.