Co-Principal Investigators: Kirsten Daehler (WestEd) and Joan Heller (Heller Research Associates)

Purpose: In the United States, districts do not generally have coherent, coordinated professional development programs, nor are they likely to have the internal resources and capacity to meet their teachers' content-specific needs in science. Examination of the TE-MAT website (http://www.te-mat.org), a database of nearly 600 professional development resources in mathematics and science run by the National Science Teacher Association's NSTA Recommends program, reveals that only a handful attend to deepening teacher's content knowledge, and a rare few consider the critical overlap between science content and classroom practice that is needed to develop teachers' pedagogical content knowledge. Given this need, WestEd developed the Making Sense of SCIENCE professional development program. The purpose of this study is to test the efficacy of this program. The study evaluates the program's series of physical science courses for middle school teachers, a coherent program that covers nearly 100 percent of the physical science standards for students. The WestEd courses are designed to help teachers: learn major concepts in science; examine how children make sense of those concepts; analyze and improve teaching practice; and support student reading, writing, and discussion in science. Each individual teacher course has been iteratively developed, field tested, and evaluated separately, showing significant impact on student learning in the elementary grades and somewhat more modest gains in middle school grades in addition to evidence of scalability. The professional development model has been revised to incorporate a substantial literacy component and hopefully improve the middle school outcomes. This study examines cumulative effects of the full series of courses.

Project Activities: In this study, researchers will test the efficacy of the Making Sense of SCIENCE program using a multi-year cluster randomized trial with the same number of schools assigned to parallel treatment and control groups. This study is intended to test the effects of the Making Sense of SCIENCE model of professional development in an urban context on grade 8 students' content knowledge in physical science (specifically in force, motion, energy, and matter) and on their science achievement generally, as measured by a state standardized test. Program effects on teachers' content knowledge in these three physical science topics will also be estimated, and their classroom practices will be characterized (specifically in relation to student-teacher interactions, attention to scientific sense-making, eliciting and incorporating students' science ideas into instruction, and integration of literacy supports around reading and writing in science). The descriptive portion of the study will examine relationships among gains in teacher knowledge, changes in classroom practice, and evidence of student learning.

Products: Products from this study will include published reports in peer-reviewed journals on the evidence of the Making Sense of SCIENCE program.

Structured Abstract

Setting: The study will take place in three urban school districts in the greater Phoenix, Arizona area.

Population: The sample will include grade 8 teachers and students. The sample will include approximately 72 middle schools, 108 teachers, and 7,640 students.

Intervention: The Making Sense of SCIENCE model of professional development consists of three 40-hour physical science courses for teachers (30-hours in a core summer course and 10-hours of school-year), followed by the Looking at Student Work formative assessment, which sought to reinforce what was learned in the summer on the topics of force, motion, energy, and matter. The professional development uses Science Investigations; carefully sequenced hands-on explorations designed to engage adult learners in known content dilemmas that surface teachers' own misconceptions and challenge their beliefs about the nature of science. Teachers also learn by reading and discussing written content notes and session reviews as a way to reinforce correct ideas. Literacy Investigations provide structure for teachers to identify and evaluate the literacy supports that guided their own learning, and reflect on the implications for their students (for example, how to present data and encourage student talk). In Science Investigations, teachers plan hands-on activities and collaborate in small-group work as opportunities for developing English proficiency in the context of authentic communication about science knowledge develop. During Teaching Investigations, teachers examine student thinking and critically analyze instruction presented in a written teaching case.

Research Design and Methods: A two-year cluster randomized trial will be conducted among grade 8 teachers and students. To allow for attrition, researchers will recruit 90 schools and grade 8 teachers across the research sites. Half of the schools at each site will be randomly assigned to a treatment group, whose teachers are eligible to participate in the Making Sense of SCIENCE professional development courses, and half will be assigned to a control group, who are excluded from the courses for 2 years, but will receive stipends for data collection. Through this design, the team will address the following research goals: (a) evaluate the efficacy of Making Sense of SCIENCE professional development courses for improving student achievement in science; (b) evaluate the promise of the program for reducing the science achievement gap between students who enter grade 8 at different ability levels; and (c) evaluate the impact of the program on teacher content knowledge in physical sciences. The team will also gather descriptive information in order to examine relationships among gains in teacher knowledge, changes in classroom practice, and evidence of student learning.

Control Condition: Control teachers and students will experience business- as-usual school years. During the academic year, they will have the opportunity to participate in any local professional development opportunities that they choose, with the exception of Making Sense of SCIENCE courses. Following the data collection phase of the study, control teachers will have the opportunity to take the courses and receive stipends for participation.

Key Measures: Key outcomes will be measured for teachers and their students during both the 2012–13 and 2013–14 school years. Student outcomes will be measured using the Misconceptions-Oriented, Standards-Based Assessment Resources (MOSART) Physical Science Test for Grade 5–8 Students and state standardized tests. Teacher outcomes will be measured using the MOSART Physical Science Test and a previously developed and tested teacher survey. Teacher interviews and classroom observations will also be conducted using the Horizon Research Inc.'s Classroom Observation Protocol.

Data Analytic Strategy: Multilevel linear regression models will be estimated to provide evidence related to the research hypotheses for each year of the study. Post-intervention outcomes for students and teachers in the intervention group will be compared to the outcomes for their counterparts in the control group. The primary hypothesis-testing analyses will involve fitting multilevel linear regression models, with random intercepts to account for the nesting of individuals within higher units of aggregation.

Project Website: http://we-msspdstudy.weebly.com/

** This project was submitted to and funded under Teacher Quality: Mathematics and Science Education in FY 2011.