Previous Award Number: R305A130060
Previous Awardee: Johns Hopkins University

Co-Principal Investigator: Celene Domitrovich (Pennsylvania State University)

Purpose: A growing number of school-based prevention programs have demonstrated positive effects on a range of educational and behavioral outcomes, yet there is considerable variation in the fidelity with which these programs are implemented. There is growing interest in coaching as a support system for optimizing implementation, which in turn could maximize outcomes for students. The primary goal of this exploratory study is to identify teacher, classroom, and school contextual factors, as well as aspects of the coaching process that influence implementation of social-emotional learning and classroom management programs. In future research, the team will build on this foundation to develop a tailored coaching model and provide information about ways to best use costly coaching resources and optimize implementation.

Project Activities: This study builds on a recently completed IES Efficacy study (A Randomized Controlled Trial of the Combination of Two Preventive Interventions) that tested the integration of the Promoting Alternative Thinking Strategies (PATHS) social-emotional learning program and the PAX/Good Behavior Game (PAX/GBG) classroom management program, referred to as "PATHS to PAX." Teachers in intervention schools received program training, onsite support and technical assistance from a coach to increase program implementation. Using existing data from the efficacy trial, the researchers will examine variation in implementation quality and dosage as a function of teacher, classroom, and school factors and identify aspects of the dynamic coaching process that are associated with high implementation quality and dosage.

Products: The products of this project will be identification of ways to improve implementation of social-emotional and classroom management programs. In particular, the team will identify whether certain features of the coaching process, matched with certain teacher, classroom, and/or school factors, contribute to high fidelity program implementation. Peer-reviewed publications will also be produced.

Structured Abstract

Setting: Elementary schools located in urban, low socioeconomic communities in Maryland participated in the efficacy study.

Sample: Data on 203 teachers across 18 elementary schools that were randomly assigned to either the PAX/GBG alone condition (9 schools) or to the PATHS to PAX condition (9 schools) will be analyzed for this exploratory study. About 40 percent of these teachers were age 30 or under when they participated in the efficacy trial and 90 percent are female. On average, this group of teachers had 11.5 years of teaching experience and 46 percent had a master's degree. The participating schools had an average enrollment of 358 students in kindergarten through fifth grade (88 percent African American and 85 percent receiving free or reduced priced meals).

Intervention: PAX/GBG is based on social learning principles and provides teachers with an efficient means of managing student aggressive/disruptive and off-task behavior via reinforcing the inhibition of these behaviors within a game-like context. PATHS seeks to reduce aggressive/disruptive and off-task behaviors via teacher-led instruction aimed at facilitating emotion regulation, self-control, social problem-solving, and conflict resolution skills. For the efficacy trial, researchers combined the PAX/GBG with PATHS—"PATHS to PAX"—to determine whether increasing attention to task and reducing disruptive behavior in the classroom through PAX/GBG strategies would facilitate the acquisition of the emotion regulation, social problem-solving, and conflict resolution skills taught in PATHS.

Research Design and Methods: In the efficacy trial, 27 schools were randomly assigned to one of three conditions: PAX/GBG alone, PATHS to PAX, or a business as usual control condition. A total of 203 teachers within the intervention condition schools (PAX/GBG and PATHS to PAX) provided data on a variety of individual, classroom, and school factors over the course of a single school year. Data on implementation quality and a three-phase coaching process were collected through coach logs. In Phase 1 (November of the school year), coaches modeled the lessons and strategies for teachers immediately following training in either PAX/GBG (one full day) or PATHS to PAX (two full days). In Phase 2 (December), coaches observed teachers' implementation in the classroom and provided feedback. In Phase 3 (January-June), coaches continued to model and/or provide additional feedback following observation for teachers who either requested additional help or for those teachers deemed by the coaches to need additional help.

The primary objective of this Exploration study is to identify teacher, classroom, and school contextual factors, as well as aspects of the coaching process (e.g., amount, type, or pattern of support), that are associated with high fidelity implementation of two commonly used social-emotional learning and classroom management programs. The data collected during the efficacy study described above will be analyzed using latent variable modeling with the primary outcomes of interest being program dosage and quality of implementation. In Year 1, the research team will use latent growth model analyses to understand how implementation quality and dosage vary as a function of teacher, classroom, and school factors. In Year 2, the team will build on the Year 1 analyses and conduct analyses focused on identifying aspects of the dynamic coaching process that are associated with high implementation quality and dosage. To assess whether the associations between teacher, classroom and school factors and implementation vary as a function of level or type of coaching, the team will model coaching as a moderator.

Control Condition: Data from the control school teachers (a business as usual condition) will not be analyzed as part of this exploratory study.

Key Measures: Data on implementation quality was collected using implementation rubrics during live observations. Data on implementation dosage was collected using weekly logs filled out by the teachers. A coach visit log was developed to measure the duration and type (e.g., face, phone, email) of coaching contact along with the specific activities performed during the contact (e.g., check in, modeling). The working relationship between the coach and teacher were assessed with alliance scales (e.g., "this teacher is open to the suggestions I provide" or "the time I spend with my coach is effective and productive"). Teachers' perceptions of the interventions was also assessed (e.g., initial efficacy, ease of use, fit with schedule), along with satisfaction (e.g., how pleased are you with this program?) and motivation (e.g., how motivated are you to continue using this program?). Coaches were also asked to rate the extent of administrator support for the programs. Teacher characteristics were also assessed, including basic demographic and professional characteristics (e.g., years teaching, use of other social-emotional programs), self-efficacy (e.g., the Behavior Management Self-Efficacy Scale), burnout (Maslach Burnout Inventory), mindfulness (the Teacher Mindfulness Questionnaire), and teachers' management style and behavior (coach ratings). Student outcomes collected included: demographic characteristics, teacher ratings of behavior (the Teacher Observation of Classroom Adaptation-Revised) and academic engagement, and school record data (e.g., standardized test scores, special education service use). Classroom characteristics were assessed via archival data (e.g., student-teacher and boy-girl ratios). School characteristics were assessed using school climate ratings (e.g., the Openness to Innovation scale) and the Organizational Health Inventory.

Data Analytic Strategy: A variety of modeling procedures (e.g., latent class/profile analysis, latent growth modeling, and structural equation modeling with continuous, dichotomous, and count outcomes) will be performed as appropriate to address two specific research questions: (1) Does implementation quality and dosage vary as a function of teacher, classroom, and school factors? and (2) What aspects of the dynamic coaching process are associated with high implementation quality and dosage? When possible, the researchers will examine potential differences by intervention condition (i.e., PAX/GBG only vs. PATHs to PAX) in order to determine if the effects vary by the type of program used. The primary outcomes for the proposed analyses are the program dosage (e.g., weekly recording of the PATHS lessons completed, the total number of games and minutes the PAX/GBG was played during the day), and quality of implementation of the games and PATHS lessons as reflected in the four rubric observations carried out bi-monthly over the school year.

Related IES Projects: A Randomized Controlled Trial of the Combination of Two Preventive Interventions (R305A080326)

Products and Publications

Journal article, monograph, or newsletter

Becker, K.D., Bradshaw, C.P., Domitrovich, C., and Ialongo, N.S. (2013). Coaching Teachers to Improve Implementation of the Good Behavior Game. Administration and Policy in Mental Health and Mental Health Services Research, 40(6): 482–493.

Berg, J. K. Bradshaw, C.P., Jo, B. and Ialongo, N. (2016). Using Complier Average Causal Effect Estimation to Determine the Impacts of the Good Behavior Game Preventive Intervention on Teacher Implementers. Administration and Policy in Mental Health and Mental Health Services Research, 44(4): 558–571.

Domitrovich, C. E., Pas, E. T., Bradshaw, C. P., Becker, K. D., Keperling, J., Embry, D., and Ialongo, N. (2015). Individual and School Organizational Factors that Influence Implementation of the PAX Good Behavior Game Intervention. Prevention Science, 16(8): 1064–1074.

Domitrovich, C., Bradshaw, C. P., Berg, J., Pas, E. T., Becker, K., Musci, R., Embry, D. D. and Ialongo, N. (2016). How Do School-Based Prevention Programs Impact Teachers? Findings from a Randomized Trial of an Integrated Classroom Management and Social-Emotional Program. Prevention Science, 17(3): 325–337.

Domitrovich, C.E., Bradshaw, C.P., Greenberg, M.T., Embry, D., Poduska, J.M., and Ialongo, N.S. (2010). Integrated Models of School-Based Prevention: Logic and Theory. Psychology in the Schools, 47(1): 71–88.

Johnson, S., Pas, E. T., and Bradshaw, C. P. (2016). Understanding and Measuring Coach–Teacher Alliance: A Glimpse Inside the 'Black Box'. Prevention Science, 17(4): 439–449.

Johnson, S.R., Pas, E.T., Bradshaw, C.P., and Ialongo, N.S. (2017). Promoting Teachers' Implementation of Classroom-Based Prevention Programming Through Coaching: The Mediating Role of the Coach–Teacher Relationship. Administration and Policy in Mental Health and Mental Health Services Research, 1–13.

Johnson, S., Pas, E. T., Loh, D., Debnam, K. and Bradshaw, C. P. (2017). High School Teachers' Openness to Adopting New Practices: The Role of Personal Resources and Organizational Climate. School Mental Health, 9(1): 16–27.

Pas, E.T., Bradshaw, C.P., Becker, K., Domitrovich, C., Berg, J., Musci, R., and Ialongo, N. (2015). Trajectories for Coaching Dosage as a Means for Improving Implementation of the Good Behavior Game. School Mental Health, 7: 61–73.

Pas, E.T., Larson, K.E., Reinke, W.M., Herman, K.C., and Bradshaw, C.P. (2016). Implementation and Acceptability of an Adapted Classroom Check-Up Coaching Model to Promote Culturally Responsive Classroom Management. Education and Treatment of Children, 39(4), 467–491.

Smith, E.P., and Bradshaw, C.P. (2017). Promoting Nurturing Environments in Afterschool Settings. Clinical Child and Family Psychology Review, 20(2), 117–126.