Project Activities
The researchers will randomly assign middle school teachers in three cohorts to receive individualized coaching through Double Check Online or self-service access to Double Check content via a learning management system. The researchers will assess teachers and students two times in the first school year (pre- and post-intervention) and again at the end of the second school year to determine immediate and longer-term effects. They will use integrative data analysis to harmonize data from this trial with prior RCTs of Double Check to examine mediators, moderators, and delivery characteristics of Double Check. They will also estimate the costs associated with program implementation in relation to outcomes and leverage extant data from the prior RCTs and cost tools to determine the relative benefits of in-person versus online delivery.
Structured Abstract
Setting
This study will be conducted in approximately 20 urban, suburban, and rural middle schools in Maryland, Missouri, and Pennsylvania.
Sample
Approximately 300 middle school teachers (grades 6 to 8) and their students (about 15 per classroom) will participate in this study.
The original Double Check program was created to provide teachers with professional development and coaching on culturally responsive practices and to promote student engagement. Double Check Online uses an enhanced version of the Classroom Check-up, which includes motivational strategies such as personalized feedback on classroom behaviors, encouraging personal responsibility for decision-making, developing a menu of options for interventions, and supporting teacher self-efficacy. Double Check Online includes professional development modules consistent with the five CARES domains (Connection to curriculum, Authentic relationships, Reflective thinking, Effective communication, and Sensitivity to students' culture) ) and evidence-based classroom management strategies. Double Check Online is fully functional, free and has been tested for feasibility and acceptability through the IES-funded National Center for Rural School Mental Health (NCRSMH).
Research design and methods
The researchers will randomly assign teachers in three cohorts to receive tailored coaching through Double Check Online (active coaching in phase 1 and sustainability activities in phase 2) or self-service access to Double Check content via a learning management system. They will assess teachers and their students in the fall and spring of the first school year to determine immediate impact and in the spring of the subsequent school year to determine sustainability.
Control condition
Teachers randomly assigned to the comparison condition will receive self-service access to Double Check content but no coaching or tailoring of the professional development or program content.
Key measures
The researchers will assess student outcomes using school archival data (office discipline referrals, suspensions, expulsions, attendance, special education referrals, and academic performance) and observations of engagement and on-task behavior using the ASSIST observation tool. They will assess teacher outcomes using teacher self-reports of culturally responsive practices using the IES-funded CARES360 Teacher Self-Reflection Tool, the Multicultural Efficacy Scale, and the Culturally Responsive Teaching Self-Efficacy Scale and using observations with the ASSIST and the IES-funded CARES360 observation tool. They will assess implementation fidelity using coaching logs and the Classroom Check-up Fidelity Checklist. They will also assess participants' satisfaction with coaching using both the Teacher-reported and Coach-reported Alliance Scales.
Data analytic strategy
To examine impacts on students and teachers, the researchers will estimate main and moderated intent to treat effects using multilevel growth models. They will use mixed methods analyses to examine factors influencing implementation. To examine the relative benefits of online versus in-person delivery and mediators and moderators impacting outcomes and fidelity, they will harmonize data from this study with extant data from prior RCTs of Double Check and analyze the data using integrative data analysis.
Cost analysis strategy
The researchers will use the NCRSMH Cost Calculator and the ingredients method to enumerate the costs derived from fidelity data, coaching log data, and data captured by the online systems in both conditions, taking into consideration opportunity costs. The final steps in the cost analysis will be to determine, for each school, how costs are distributed across levels or stakeholders such as individual teachers, administrators, districts, and implementing partners, and to enumerate costs in relation to benefits using shadow pricing.
People and institutions involved
IES program contact(s)
Project contributors
Products and publications
Products: This project will result in evidence of the efficacy of an online version of Double Check, a professional development and coaching program to support culturally responsive practices, for middle schools and information about the costs of implementing Double Check online and the potential cost benefits of the online delivery compared to in person training and coaching. The project will also result in a final dataset to be shared, peer-reviewed publications and presentations, and additional dissemination products including book chapters, blogs, and briefs. These materials will be disseminated to researchers and practitioners through the University of Virginia (UVA) Youth-Nex Center for Positive Youth Development and the state-wide UVA K-12 Advisory Council, which includes multiple local education agencies in Virginia.
ERIC Citations: Find available citations in ERIC for this award here.
Related projects
Supplemental information
Co-Principal Investigators: Bottiani, Jessika; Debnam, Katrina
Questions about this project?
To answer additional questions about this project or provide feedback, please contact the program officer.
