Co-Principal Investigator: Iovannone, Rose; Ginns, Diana

Purpose: The purpose of this project is to develop MOTIVATED, a modular-based consultation coaching framework to help elementary school teachers select and implement evidence based, class-wide behavior interventions for students with emotional and behavioral disorders (EBD) who are placed in self-contained classrooms. The number of students who are found eligible for special education services under the eligibility of emotional disturbance (ED) has been steadily increasing over the past 40 years. Unfortunately, teachers of students with ED often are undertrained or unable to access evidence-based interventions to use in their classrooms. The current project intends to address this by developing a collaborative consultation framework to assist teachers in selecting, implementing, and evaluating evidence-based behavior practices matched to their classroom needs. The research team will iteratively develop and pilot test MOTIVATED to promote teacher implementation fidelity of evidence-based interventions that leads to improved student behavior and academic functioning.

Project Activities: The research team will develop and pilot test the intervention over the course of 3 years. During the first year of the project, the team will focus on development of MOTIVATED materials and processes that will be refined based on input from stakeholder focus groups and expert panel reviewers. The completely developed intervention will be tested for its feasibility, functionality, and usefulness in Year 2, using a single case, multiple baseline design across five classrooms. Feedback from end users (teachers) will be sought to refine the intervention as necessary and to ready it for a pilot evaluation. During Year 3, an underpowered randomized controlled trial will be used to determine the promise of the refined MOTIVATED for improving teacher implementation of evidence-based interventions and teacher self-efficacy, and student academic engagement and behavioral outcomes. Researchers will also examine the costs of implementing the version of the intervention tested in the pilot study.

Products: This project will result in a fully developed intervention framework that will be feasible for use by teachers in selecting, implementing, and evaluating evidence-based class-wide behavior interventions for students with ED who are placed in self-contained settings. The framework will include assessments, decision-making guides for selecting and prioritizing class-wide interventions, protocols and templates for customizing selected practices to meet the contextual needs of classrooms, coaching manuals and processes to support teacher implementation of modular interventions, and problem-solving guidelines for evaluating effectiveness. The project will result in peer-reviewed publications and presentations as well as additional dissemination products that reach education stakeholders such as practitioners and policymakers.

Structured Abstract

Setting: The research will take place in elementary schools in Florida.

Sample: The primary sample will include teachers of students with ED who are placed in self-contained settings. In year 1, approximately 72-96 educators, other school personnel, and students with ED will participate in focus groups that will assist in the module development and refinement. During year 2, we will enroll five teachers of self-contained classrooms for students with ED and approximately 50 students in the classrooms to assess the feasibility, functionality, and usefulness of MOTIVATED. In year 3, the study will enroll 16 teachers of self-contained classrooms for students with ED and approximately 160 students in those classrooms. During all three years of the project, an advisory board consisting of three national experts and two experienced practitioners will provide expertise related to module development to the project team.

Intervention: The current literature highlights over 20 empirically supported practices for class-wide behavior management for students with ED across 5 broad categories. MOTIVATED is a modular-based framework that will provide educators a process for assessing, selecting, implementing and evaluating class-wide behavioral strategies that best match the context and needs of the classroom. Modular approaches have been shown to be an efficient method for enhancing implementation of interventions. Rather than requiring teachers to learn more than twenty intervention strategies, some of which are not suitable for specific contexts and needs, a modular approach saves resources by prioritizing 2-4 interventions that teachers will be trained to implement and are a best match for classroom concerns. MOTIVATED will be comprehensive and will include (a) teaming processes, (b) classroom assessments to identify areas of strength and need related to class-wide behavioral practices, (c) completely developed modules for each intervention that include templates for implementation procedures that are customizable with vignettes and supplemental materials, (d) a coaching process that will use teacher goal-setting and behavior skills training to ensure that teachers are taught the skills to implement the interventions with fidelity and make necessary adjustments contingent upon teacher reflection, performance feedback by the coach, and data trends, and (e) structured evaluation meetings to make next-step decisions based on data.

Research Design and Methods: The research will be conducted across 3 years. During the first year of the project, the team will use qualitative methodology including stakeholder focus groups and expert reviewers to get input on MOTIVATED processes and materials. In Year 2 the research team will conduct a feasibility study using a single case, multiple baseline design across five classrooms, and the intervention will be refined based of the results and input from end users. During Year 3, an underpowered randomized control trial across 16 elementary school classrooms (8 intervention and 8 control classrooms) will be used to assess the intervention's promise of teacher implementation fidelity of selected modular interventions, teacher self-efficacy, and student academic engagement and behavioral outcomes. Researchers will also conduct a cost analysis of implementing the version of the intervention tested in the pilot study

Control Condition: In the Year 3 pilot study, schools randomized to the services-as-usual condition will receive the typical behavioral services and supports provided to teachers of students with ED in self-contained settings.

Key Measures: Measures will assess intervention fidelity, feasibility, and acceptability as well as team-, teacher-, and student-level outcomes. At the teacher level, the Generalized Intervention Fidelity Form – Coach-Rated will be used to assess fidelity, the Usage Rating Profile – Intervention (Revised) will assess feasibility, acceptability, usefulness, and functionality of the intervention The Teacher's Sense of Efficacy Scale will be used to assess teacher self-efficacy. Student behavior outcomes will be assessed with Direct Behavior Ratings, and systematic direct observation. Academic engagement will be measured using The Engagement Check II. The Woodcock-Johnson IV Tests of Achievement will be used to measure student academic outcomes. Office discipline referrals, out-of-school suspensions, in-school suspensions, and time spent in general education will be obtained as measures of student outcomes.

Data Analytic Strategy: Qualitative methods in year 1 will be analyzed for themes using MAXQDA software, which allows for coding and theme identification. Expert panel review input will be analyzed by computing a content validity index (CVI) for each scaled item in the structured questionnaire and theme identification for any open-ended comments. In year 2, a single case multiple baseline design will be implemented and will incorporate assigning random classroom start times and on-going visual analysis of the data to guide the timing of interventions. A treatment effect inference will be conducted by using masked visual analyses in which behavior analysts naïve to the start of intervention of each classroom will review graphs of time-series data with unmarked phases and attempt to determine the order that each classroom began the intervention. Effect sizes will be calculated using multilevel models to estimate individual treatment effects and average treatment effect along with TauU as an additional measure of effect for each teacher. During year 3, the pilot study data will be analyzed using a separate statistical model for each outcome post-intervention, including mixed linear models for continuous student outcomes, generalized mixed linear models for count-based student outcomes and standard linear model for teacher outcomes. A cost analysis employing an ingredients method will be conducted to determine start-up costs and recurring costs to implement the program at the school, classroom, and individual student level. In years 2 and 3, descriptive data (e.g., means, SDs) will be used to evaluate teacher self-efficacy and social validity.