Co-Principal Investigators: Sutherland, Kevin

Purpose: The purpose of this project was to develop a web-based version of an existing tier 2 intervention, Behavioral, Emotional, and Social Training: Competent Learners Achieving School Success (BEST in CLASS), that targets the reduction of problem behavior of young children at risk for emotional and behavioral disorders (EBD). BEST in CLASS was designed to be delivered face-to-face in early childhood classrooms, which may restrict accessibility for early childhood programs with limited personnel or financial resources to support the program, or for programs in geographically remote locations where face-to-face interventions face logistical challenges. In addition, on-site delivery of professional development may be less convenient for busy teachers. To respond to these needs, the research team developed a web-based version of the BEST in CLASS intervention (BEST in CLASS — Web) to increase the accessibility, flexibility, usability, and scalability of the intervention for early childhood teachers working with young children at risk for EBD.

Project Activities: The team used an iterative process to develop BEST in CLASS — Web. In Year 1, the researchers modified BEST in CLASS training and coaching materials based on feedback from teachers and families. They also developed the website on which the intervention is hosted. In Year 2, the researchers trained and coached teachers to use this preliminary version of BEST in CLASS — Web and gathered feedback from teachers to learn more about the usability, feasibility, and promise of the web-based version. In Year 3, the researchers conducted a pilot test of BEST in CLASS — Web by randomly assigning teachers to the BEST in CLASS — Web or original BEST in CLASS condition to determine the promise of the web-based adaptation compared to the original version for improving teacher practice and student behavior and academic performance.

Structured Abstract

Setting: This study took place in early childhood programs and centers in Florida and Virginia.

Sample: In the initial phase, 13 teachers (8 teachers with previous BEST in CLASS intervention experience and 5 without BEST in CLASS experience) participated in a focus group and 7 early childhood program administrators participated in structured interviews. Seven pilot users from local early childhood teacher preparation programs participated in usability and feasibility testing over the first 2 years. Nineteen early childhood teachers and 34 children ages 3-5 who were at risk for EBD participated in the pilot study. A third group of 10 teachers, along with 20 children, were randomly selected from a pool of comparison participants from a previous BEST in CLASS efficacy trial to serve as a business-as-usual (BAU) control group.

Intervention: BEST in CLASS — Web is a modified version of the original BEST in CLASS, which was developed and tested for efficacy with previous IES funding. BEST in CLASS is a strengths-based, manualized intervention model that targets the prevention and improvement of problem behaviors demonstrated by young children at risk for EBD in early childhood classrooms. This intervention combines evidence-based practices with a practice-based coaching model designed to increase the use of the practices by teachers of preschool-age children. Training consists of a 6-hour workshop that uses didactic and interactive learning activities supported by video examples of and opportunities to demonstrate effective practices. For the BEST in CLASS — Web intervention, all training components are delivered via eight web-based modules. Teachers receive a training manual (with modules on evidence-based practices to improve and prevent problem behaviors, increase engagement and social skills, and develop positive teacher-child interactions) as well as 14 weeks of practice-based coaching through video conferences designed to increase the quality and quantity of their use of BEST in CLASS practices. In addition to teacher use of effective practices, BEST in CLASS aims increase teacher self-efficacy for enhancing the quality of teacher-child relationships and the classroom environment.

Research Design and Methods: In phase 1, the researchers adapted the BEST in CLASS training and practice-based coaching materials based on focus groups of teachers and interviews with program administrators. Training and service delivery materials were adapted to make them appropriate for web-based usage and adaptable for early childhood programs with differing technological capacities. The research team also developed the website for BEST in CLASS to be delivered online. In phase 2, the researchers further refined the BEST in CLASS website and materials and conducted usability and feasibility studies in order to prepare for a full version for pilot testing. Feedback from teachers as well as teacher and child outcome data were used to inform revisions to the model. In phase 3, the researchers conducted a multisite cluster randomized pilot study to determine the promise of BEST in CLASS — Web for improving teacher practice and behavioral and academic-related outcomes of children at risk for emotional/behavioral disabilities.

Comparison Conditions: During the pilot study, there were two comparison groups — teachers received either the BEST in CLASS on-site (original in-person intervention) or business-as-usual (BAU) conditions. The teachers in the BAU group were randomly selected from the control group of a previous BEST in CLASS efficacy study.

Key Measures: The following key measures were used to examine teacher outcomes: Classroom Assessment Scoring System, Teacher Self-Efficacy Scale, Teacher—Child Interaction Direct Observation System, and researcher-developed scales to measure technology use and self-efficacy and attitudes toward coaching. The following key measures were used to examine child outcomes: Caregiver/Teacher Report Form, Social Skills Improvement System Rating Scales, Student Teacher Relationship Scale, and the Preschool Self-Regulation Assessment. In addition, a variety of measures were used to examine implementation fidelity (coach and teacher), including observational measures, rating scales, and surveys. Finally, teachers completed an acceptability measure and children's families completed a social validity measure.

Data Analytic Strategy: The researchers used qualitative data analytic techniques to analyze the focus group and interview data to identify emerging patterns and themes during the first 2 years. In the second year, the researchers conducted descriptive analyses to identify any changes in teacher and child outcomes from baseline to post-intervention and to examine fidelity and usability. For the randomized pilot study, the research team used analysis of variance (ANOVA) methods, with repeated measures for comparisons within subjects over time and randomized ANOVA methods for mean between-group comparisons. The team also conducted cost comparisons to determine the cost differences between BEST in CLASS — Web and BEST in CLASS onsite.

Related IES Projects:
Efficacy of BEST in CLASS Intervention for Young Children at High Risk for Emotional and
Behavioral Disorders (R324A110173); Promoting Social, Emotional, and Behavioral Competence in Young High-Risk Children: A Preventative Classroom-based Early Intervention Model (R324A080074); A Conceptual Replication of BEST in CLASS: An Efficacy Study of BEST in CLASS-Elementary (R305A180182); BEST in CLASS - Elementary: A Preventative Classroom-based Intervention Model (R305A150246)

Products and Publications

ERIC Citations: Find available citations in ERIC for this award here.

Project Websites:
https://education.ufl.edu/best-in-class/about/about-us/
https://bestinclass.ori.org/login/

Additional online resources and information:
https://ceecs.education.ufl.edu/b-behavior/
https://ceecs.education.ufl.edu/d-discipline/
https://ceecs.education.ufl.edu/p-practice-based-coaching/