Previous Award Number: R305A210241
Previous Awardee: University of Minnesota

Co-Principal Investigator: Pullmann, Michael

Purpose: The purpose of this project is to test the effects of BASIS-T (Beliefs and Attitudes to Support Implementation in Schools – Teachers) on the implementation and student outcomes of evidence-based prevention programs (EBPP) via a hybrid type 3 implementation-effectiveness trial. Failing to address individual-level motivational factors of implementers is especially costly, as individual behavior change is ultimately required for successful EBPP implementation, even when organizational factors such as evidence-informed policy, supportive leadership, and effective training are in place. Hybrid type 3 trials allow for testing the effects of implementation strategies, as well as observing the reproducibility of particular EBPPs — in this case, Positive Greetings at the Door (PGD), an intervention focused on educators greeting children at the door to improve social and emotional outcomes.

Project Activities: During the first year of this project, the research team will secure IRB approval, solidify partnerships with participating schools and teachers, and engage in planning to ensure that the project is minimally disruptive to everyday educational service delivery yet maintains rigor to test the effects of BASIS-T on implementation and student outcomes. Subsequent years of the project will involve (a) blocking schools into pairs and randomly assigning schools within pairs to the BASIS-T+PGD or the Active Comparison Control+PGD conditions, (b) delivering the BASIS-T implementation strategy and supporting the delivery of PGD training and consultation to participating teachers over the year, (c) gathering data across multiple time points, and (d) following teachers into a second year to examine to what extent teachers sustain implementation and whether sustained implementation of PGD leads to improved student outcomes.

Products: Based on this initial efficacy trial, the research team will provide findings that highlight whether the BASIS-T group-based, motivational implementation strategy increases the yield of training and consultation on implementation and student outcomes. Findings will inform more strategic efforts to bridge the gap between education research and practice thereby decreasing 'research waste' in education, which results when evidence-based programs and practices are not adopted and implemented in everyday school settings and ultimately students are unable to receive and benefit from them.

Structured Abstract

Setting: The planned studies will occur in 46 schools located in different geographic regions (rural, suburban, and urban) that serve diverse populations of students in the state of Minnesota with student demographics representative of schools in the US. Half of the schools will be assigned to the BASIS-T+PGD condition and the other half to the ACC+PGD condition.

Sample: This project will include roughly 6 teachers and 6–8 student participants from each of the participating schools, resulting in roughly 276 teachers and classrooms of students across the two study conditions.

Intervention: This efficacy project includes two different interventions. The first intervention is BASIS-T (Beliefs and Attitudes to Support Implementation in Schools - Teachers) which is an implementation intervention that is designed as an adjunct to EBP training and consultation. It includes a pre-training session that is designed to increase teacher motivation prior to training, a post-training session that is designed to support teachers to create and enact plans that are aligned with their motivation to implement, and a booster session that is designed to buffer against motivational dips that teachers may experience when trying to implement a new practice. The intervention that students receive in this study is Positive Greetings at the Door (PGD), which is a way for teachers to start class or the day with a positive interaction such as by greeting students by name, shaking a student's hand, asking a short question, or making a friendly comment that communicates they are welcome. Prior research indicates that this simple routine can improve academic engagement, decrease unwanted behavior, and improve student-teacher relationships.

Research Design and Methods: This efficacy study will use a hybrid type 3 implementation-effectiveness trial. Specifically, this hybrid type 3 trial will use a blocked randomized cohort design with an active comparison control condition to rigorously test the effects of BASIS-T on implementation and student outcomes, as well as examine with whom, under what conditions, and how/why BASIS-T works to improve implementation and student outcomes. During the project period, this study will be registered in ClinicalTrials.gov (ID: NCT05989568).

Control Condition: Teachers randomly assigned to the ACC condition will receive a 3-hour pre-training session, a 90-minute post-training session, and an online booster 15-days post training to mirror the duration of BASIS-T. The ACC condition provides support to help teachers to understand EBPPs generally and the core components of PGD, as well as controls for dose and information provided.

Key Measures: The project team will use a variety of measures to assess implementation outcomes, student outcomes, and mediator and moderator variables. They will gather implementation outcome data via direct observation using an established fidelity instrument and monthly teacher self-reports of fidelity. They will gather student outcome data via direct observations of student classroom behavior, teacher-completed class wide direct behavior rating, and administrative records of discipline and attendance. They will assess mediators via established measures of theoretical mechanisms of behavior change such as attitudes, self-efficacy, and intentions to implement. They will assess moderator variables using teacher level measures of baseline functioning and demographics and organizational level characteristics such as implementation leadership and climate.

Data Analytic Strategy: The researchers will use 3-level mixed effect model to test main effects of BASIS-T on mechanisms of behavior change, implementation outcomes, and student outcomes. They will analyze mediation using path analysis to determine whether (a) intentions mediate the relationship between BASIS-T and implementation outcomes and (b) whether implementation outcomes mediate the relationship between BASIS-T and student outcomes. They will analyze moderation by including teacher- and school-level interaction terms to examine whether they moderate the effects of BASIS-T on implementation and student outcomes. Last, they will use a qualitative approach (directed content analysis) to understand teachers whose implementation behavior does not fit with the underlying theory of change and who are unaccounted for by the mediation model.

Cost Analysis: This team will use the "ingredients approach" specifically the CostOut program to complete the cost analysis for BASIS-T and PGD implementation-as-usual (excluding ACC costs). CostOut allows users to specify the ingredients for each intervention condition, assign prices (national and user-inputted local values), and calculate direct and indirect costs based on the units per ingredient used in each condition.

Related IES Projects: Development and Evaluation of the Beliefs and Attitudes for Successful Implementation in Schools for Teachers (BASIS-T) (R305A170292)