Education Agency Partner: Lawrence Public Schools and Topeka Public Schools, Kansas

Co-Principal Investigators: Buckman, Mark Matthew; Oakes, Wendy Peia; Common, Eric Alan

Purpose: This project aims to (1) analyze existing data to determine how internalizing and externalizing behavior patterns, as well as referrals for special education eligibility, may have shifted over time with the pandemic, and (2) test Recognize. Relax. Record. (RRR). RRR is an intervention designed to reduce symptoms of anxiety, increase engagement, maximize learning recovery, and improve academic outcomes for students with and at risk for emotional and behavioral disorders (EBD). Due to the COVID-19 pandemic, schools are in immediate need of actionable and focused research-based strategies to support students' academic, behavior, and social development. One particularly vulnerable population is students with or at risk for EBD, particularly those with elevated internalizing behavior patterns. Given the educational complexities and personal trauma experienced during the pandemic era, many students are expected to exhibit elevated levels of internalizing issues (such as anxiety), which may impede learning. At the same time, teachers are striving to maximize student engagement to facilitate learning and well-being. This project aims to better understand the impact of the pandemic on students' social and emotional needs and provide schools with an intervention to address the expected rise in students with internalizing behavior challenges.

Project Activities: To evaluate the impact the pandemic has had on the internalizing behavior patterns of elementary students (Aim 1), the research team will analyze extant and new K–5 data from school districts with diverse student populations. A series of single case design (SCD) studies followed by a randomized controlled trail (RCT) will be used to better understanding implementation and efficacy of RRR to decrease symptoms of anxiety and increase engagement in students with and at risk for EBD (Aim 2).

Products: Products for this project will include information for educators and families on (a) changes in internalizing and externalizing behavior in students over the course of the pandemic (outcome of Aim 1) and (b) the efficacy of RRR to support students with internalizing behaviors (outcome of Aim 2). They will also include publications in peer-reviewed and practice-oriented journals, presentations, workshops, and online videos, as well as products that support use of evidence from this project by education practitioners.

Structured Abstract

Sample: For Aim 1, data from approximately 8,000 elementary students in 22 schools in two diverse districts will be analyzed. For Aim 2, the first SCD includes two 5th grade teachers and four students with or at risk for EBD per classroom and the second SCD involves two 4th or 5th grade teachers and four students with or at risk for EBD per classroom. For the RCT, approximately 64 teachers and 192 students in fourth and fifth grades with or at risk for EBD will participate.

Intervention: RRR is a previously developed Tier 2 intervention designed to be used by elementary classroom teachers. It is comprised of three primary components, each addressed through explicit instruction: (a) Recognize (students learn to identify thoughts and feelings related to being anxious), (b) Relax (students learn relaxation strategies to support self-regulation), and (c) Record (students learn to self-monitor their thoughts and feelings, their use of the selected relaxation strategies, and their sense of engagement in activities). RRR also includes a professional learning module to assist educators in learning about and using the intervention to support their students. The intervention is provided over 6–12 individual sessions, each lasting 15–20 minutes.

Research Design and Methods: For Aim 1 (Year 1) various multilevel models will be used to determine the impact of cohort (pre-pandemic and recent) on internalizing and externalizing behaviors in fall of each year and for the change over each year, as well as to determine the impact of cohort on outcomes such as reading scores and special education referrals. Researchers will also examine whether results vary by groups of students within the sample, including students with disabilities, students of color, and grade level of the student. For Aim 2, in Year 1, two multiple-baseline SCD studies will be implemented with research staff as interventionists. In Year 2, the SCD studies from Year 1 will be repeated but with teachers as interventionists. In Year 3, teachers will be randomly assigned to either the treatment condition or a waitlist control condition. Within each classroom, three students with a moderate level of internalizing behaviors based on screening data will be included in the study. Data will be gathered at pre- and post-intervention as well as 4 weeks after intervention.

Control Condition: The waitlist control condition for the RCT will receive business-as-usual instruction.

Key Measures: For Aims 1 and 2, researchers will use the district-administered Student Risk Screening Scale for Internalizing and Externalizing Behavior (SRSS-IE), a systematic behavior screener for K-12 students. Administrative data for Aim 1 include school absences, suspensions, expulsions, referral for special education, and special education placement. Student demographic data regarding sex, ethnicity, race, and grade level will be collected. Student reading performance will be examined with district administered AIMSweb and the Scantron Performance Series. For Aim 2, key outcome measures include the Achenbach System of Empirically Based Assessment (teacher and parent ratings of behavior) and the Social Skills Improvement System (SSIS; teacher, parent, and student ratings) as well as academic competence on the SSIS and academic engagement using Direct Behavior Ratings. Measures of treatment integrity include checklists for all components of RRR. Teachers, parents, and students will complete respective versions of the Intervention Rating Profile-15 to measure social validity.

Data Analytic Strategy: For Aim 1, three-level longitudinal multilevel growth models (observations of students nested within schools) will determine the impact of cohort (pre- pandemic/recent) on internalizing and externalizing behaviors in fall of each year and for the slope over each year. Two-level models with students nested within schools will be used to determine the impact of cohort on reading scores and referrals to special education and to explore group differences in these outcomes. For Aim 2, visual analysis of level, trend, and variability of the data will be used to analyze the SCD studies. For the RCT, longitudinal multivariate multilevel modeling will be used to examine the effect of RRR on students' levels of anxiety, internalizing behavior, academic engagement, and academic competence with an intent to treat approach. Additional models will be developed to examine the efficacy of RRR when implemented with sufficient integrity (80% or greater) as measured by behavior component checklists.

Cost Analysis: The cost per student to implement RRR compared to BAU will be calculated using the ingredients method. The cost-effectiveness of RRR will be determined by using outcomes from the multivariate multilevel models. A cost-effectiveness acceptability curve will be constructed to help decision makers interpret results.

Related Projects: Implementing Comprehensive, Integrated, Three-tiered Models to Meet Students' Academic, Behavior, and Social Needs: A Researcher-Practitioner Partnership (R305H150018); Enhancing Ci3T: Building Professional Capacity for High Fidelity Implementation to Support Students' Educational Outcomes (Project ENHANCE) (R324N190002); Project SCREEN: Validation of a Free-Access Screening Tool for K–12 Educators to Screen Students for Internalizing and Externalizing Behavior Patterns (R324A190013)