Contexts Inside and Outside of School Walls as Predictors of Differential Effectiveness in Preschool Professional Development
Co-Principal Investigator: Dana McCoy (Harvard University)
Purpose: The goal of this project is to identify variables that create differences in impacts of early childhood professional development interventions. Achieving this aim will support decision making around scaling and refining professional development programs in order to optimize classroom quality and child learning outcomes.
Project Activities: This is a retrospective study of data already collected through the National Center for Research on Early Childhood Education Professional Development Study (NCRECE PDS). The primary activities of the grant will therefore be analyzing the data for a 4-year span (2008–2011), interpreting the results, and disseminating the findings to academic and policy audiences.
Products: The results will provide evidence of the extent to which the effects of professional development programs for early childhood education providers vary, as well as evidence regarding the extent to which relevant variables predict that variation. The project is expected to produce peer-reviewed conference presentations and journal manuscripts. The research team will also share the findings with practitioners and policy makers in the NCRECE PDS cities.
Setting: This study leverages existing data from the NCRECE PDS, which took place in schools in nine sociodemographically diverse cities in the United States, such as Chicago, IL, Stockton, CA, and Charlotte, NC.
Sample: The full analytic study sample includes 1,134 children, 263 teachers, and 163 centers in the course phase of the intervention, and 1,407 children, 326 teachers, and 198 centers in the coaching phase of the intervention. The sample, predominantly low-income and racially/ethnically diverse, is designed to represent the population of children in community and publicly-funded preschool programs (e.g., Head Start) in the United States.
Intervention: The NCRECE PDS evaluated the independent and additive impacts of two professional development components designed to enhance teacher-child interactions: (1) a 14-week course and (2) individualized coaching. The course was designed to increase in-service teachers' knowledge about the importance of teacher-child interactions to help promote children's learning and skill acquisition. The coaching occurred through a web-based system onto which teachers uploaded videos of their teaching. After watching the videos, coaches helped teachers to identify effective teaching behavior, make connections between teacher and child behaviors, and problem solve to determine alternative strategies when necessary.
Research Design and Methods: The interventions were randomly assigned to teachers via a 2x2 sequential design. In Phase I, teachers were randomly assigned either to receive the 14-week professional development course or to be in the control group. In Phase II, teachers were again randomized, this time either to receive coaching or not receive coaching. This process generated four experimental groups: (1) no course/no coaching (i.e., control); (2) course/no coaching; 3) no course/coaching; and (4) course/coaching. Findings from an intent-to-treat analysis indicated that the course and coaching, both separately and together, led to increases in teachers' use of effective social and instructional interactions. Regarding child outcomes, the course component and the coaching component separately were found to improve child outcomes for expressive language and self-regulation skills. These average treatment effects on teachers and children have provided important, highly policy relevant evidence on the utility of professional development programs. In this study, the researchers expand the evaluation of these main effects to explore variation in impacts and potential predictors of this variation via multilevel modeling.
Control Condition: Teachers in the control condition were followed across time but did not receive any professional development services.
Key Measures: Classroom quality outcomes were measured by the Classroom Assessment Scoring System™. Child language, emergent literacy skills, and self-regulation were measured using the PPVT, WJ-III Picture Vocabulary, the Phonological Awareness and Print Knowledge subtests of the Test of Preschool Early Literacy, Pencil Tap, and the Backward Digit Span. Data for contextual predictors of impact variation within and outside of school will also be used.
Data Analytic Strategy: Researchers will use multilevel modeling: to evaluate impact variation in the NCRECE PD study; to explore whether variation in treatment effects is predicted by characteristics within school walls; to explore whether variation in treatment effects is predicted by characteristics outside of school walls. To render the analyses more manageable in terms of the number of predictors in a given model, data reduction techniques, such as factor analysis and cluster analysis, will first be used to combine or eliminate predictors.