Co-Principal Investigators: Oakes, Wendy Peia; Chafouleas, Sandra; Briesch, Amy M.; Royer, David James; Common, Eric Alan

Purpose: The purpose of this project is to conduct an efficacy trial of the Comprehensive, Integrated, Three-Tiered (Ci3T) model of prevention for elementary schools that integrates academic, behavioral, and social learning supports. In addition, the research team will further identify and develop systems-level professional learning modules necessary for moving the Ci3T model to scale, including (1) Leadership Skills and Structures Needed to Support Ci3T, (2) Capacity of Ci3T Leadership Teams to Support Implementation, and (3) Data Systems for Behavior Screening. Ci3T has demonstrated promise through a series of descriptive, experimental, and psychometric studies to develop the fully integrated system, but the full Ci3T model has not yet been rigorously evaluated. The project will culminate with a pilot test of an enhanced Ci3T model to promote scalability and sustainability using the newly developed professional learning tools.

Project Activities: The research team will conduct activities across two major components. In Component 1, the research team will conduct a randomized controlled trial of the Ci3T model and evaluate its cost-effectiveness. In Component 2, the research team will use an iterative process to develop professional learning modules to enhance Ci3T in ways that improve the scalability of the model and pilot test the Enhanced Ci3T model, E-Ci3T.

Products: The products of this project will include evidence of efficacy of the Ci3T model for elementary schools for improving student academic, behavioral, and social outcomes. In addition, the project will result in a fully developed E-Ci3T model with evidence of promise for improving the treatment integrity and social validity of Tier 1 practices. Products will also include peer-reviewed publications, presentations, reports, and other products for a wide range of audiences (e.g., policymakers, practitioners, researchers).

Structured Abstract

Setting: The research will take place in elementary schools in Kansas for Component 1, and Kansas, Washington, and Vermont for Component 2.

Sample: In Component 1, 21 schools will participate. Within each school, participants include (a) Ci3T Leadership Teams (approximately 8 adults per team, including administrators, general and special educators, school counselors, and parents) and (b) the full faculty and staff. The sample for Component 2 includes 36 elementary schools currently implementing Ci3T and 5 elementary schools that will begin implementation of Ci3T. Participants include the Ci3T Leadership Team members at each school.

Intervention: Ci3T is a comprehensive, integrated, data-driven prevention model with structures for monitoring system- and student-level data to determine effectiveness in meeting system/school goals and informing academic, behavioral, and social skill instruction for students. Specifically, the Ci3T model blends the principles of a Response to Intervention framework focused on academic domains, positive behavioral interventions and supports focused on behavioral domains, and a validated program focused on developing students' social skill sets. The Ci3T model involves using universal screening data to (a) examine school-wide student performance over time and (b) detect individual students for whom primary prevention efforts are insufficient, thus requiring evidence-based strategies and practices at Tier 2 or Tier 3. A keystone feature of Ci3T is data-informed decision making. Ci3T had been highly focused on the importance of data-informed decision making, with attention to prioritizing systems, structures, and tools for universal screening of academics; however, there has been greater attention in recent years to social, emotional, and behavioral indicators. Central to Ci3T systems change efforts is high-quality, on-going, practice-based professional learning and coaching to build district- and school-site leadership and capacity to sustain Ci3T. To respond to this need, the new professional learning modules to help sustain Ci3T implementation will be developed to create the Enhanced Ci3T model.

Research Design and Methods: To evaluate the initial efficacy of the Ci3T model (Component 1), the research team will use a cluster randomized controlled trial in which schools will be randomized to intervention or a waitlist control. A matched-pair block randomization procedure will be used to ensure school characteristics are balanced across the treatment and waitlist control groups. Schools in the intervention group will design their Ci3T model in Year 1 of the project and begin implementation in Year 2. Schools in the waitlist control group will design their Ci3T model in Year 3 and begin implementation in Year 4. For Component 2, in Years 1 – 3, researchers will iteratively develop, refine, and test professional learning modules for the three focal areas to result in an Enhanced Ci3T (E-Ci3T). In Year 1, semi-structured interviews and focus groups with Ci3T Leadership Teams will be used to understand the variables and contexts influencing the implementation of Ci3T models and factors that facilitate or impede the adoption and continued use Ci3T. Similar procedures will be used for learning about the uses and challenges related to behavior screening measures and about their current use of screening tools and screening practices. In addition, a survey will be administered to all faculty, staff, and administrators in participating schools that have implemented Ci3T in the past to assess Ci3T practices currently in place and the resources and professional learning needs they view as valuable to continued Ci3T implementation for their school. In Years 2 and 3, based on the feedback collected in Year 1, various professional learning tools will be developed (such as multi-media presentations, practice guides, knowledge assessments) and iteratively tested and revised across participating sites. A quasi-experimental study comparing E-Ci3T and Ci3T (Years 4 – 5) will be used to assess the promise of E-Ci3T. Researchers will collect information on the intervention's cost and examine the intervention's cost-effectiveness. In Year 5, the research team will disseminate findings and materials to other researchers, district and school administrators, state and local decision-makers, and technical assistance providers.

Control Condition: Participating schools in Component 1 will be randomized to either receive the Ci3T program immediately (intervention condition; Year 1) or to wait and receive the intervention later (waitlist control condition; Year 3).

Key Measures: For Component 1, measures include the Ci3T Knowledge, Confidence, and Use (KCU) Survey, a researcher-developed survey for Ci3T Leadership Team members to rate their knowledge about, confidence in using, and usefulness of key constructs of the Ci3T model and the Teacher - Social Validity Survey: Primary Intervention Rating Scale (PIRS) to measure participant's views of the social significance of the intervention goals and social acceptability of the intervention procedures. Researcher-developed fidelity measures include the Ci3T Teacher Self-Report to measure teacher implementation of the Ci3T plan's procedures for instructional practices (such reinforcing and monitoring) and the School - Fidelity of Implementation - Ci3T Direct Observation to directly assess use of those instructional practices by a subset of teachers. Multiple sources of data will be used to measure student achievement outcomes (e.g., curriculum-based measures, formative assessment, state assessments) and progression through the school system (e.g., attendance, referrals for services). For Component 2, measures include researcher-created protocols for structured interviews and focus groups. The Ci3T Professional Learning Survey is a researcher-developed survey for use with school faculty, staff, and administrators to learn about existing Ci3T practices in place, topical areas of interest for professional learning, and desired avenues for accessing professional learning. The NEEDs2 Questionnaires is designed for use with faculty, staff, administrators, and parents to assess the how, which, and why of behavior screening practices implemented and if the practices impact student outcomes. In addition, the same measures of student achievement outcomes used in Component 1 will also be used for Component 2.

Data Analytic Strategy: For the randomized controlled trial in Component 1, longitudinal multilevel analyses that take into account repeated observations and nesting of students within schools will be used to examine the impact of Ci3T on teacher knowledge and use of Ci3T strategies as well as student academic and social/behavioral outcomes. For Component 2, qualitative data from focsus groups and interviews will be transcribed and analyzed using the constant comparative method to identify themes, and findings will be used to guide changes and enhancements to the professional learning modules. Survey data will be analyzed using descriptive statistics, with results also informing the content for Component 2. Costs will be estimated using the ingredients method and cost-effectiveness will be implemented using a bivariate model that acknowledges a joint distribution of costs and effects, includes random effects for school, and allows for student-level variances.

Related IES Projects: Exploring the Status and Impact of School-Based Behavior Screening Practices in a National Sample: Implications for Systems, Policy, and Research (R305A140543); Project VIABLE-II: Unified validation of Direct Behavior Rating (DBR) in a problem-solving model (R324A110017); Project VIABLE: Validation of Instruments for Assessing Behavior Longitudinally and Efficiently (R324B060014), Project EASS-E: Expanding Approaches to School Screening With Equity (R305A220249).