Co-Principal Investigator: Sutherland, Kevin

Purpose: Elementary school is an important time to promote the SEB competencies of students. Comprehensive intervention models (i.e., manualized programs that include multiple practices such as behavior specific praise) and discrete practices (i.e., a specific behavior or action) exist that improve SEB outcomes of elementary students, but these models and practices do not demonstrate the same level of efficacy across all students. As a result, it is difficult for researchers and stakeholders to access the literature to determine what models or practices have the strongest evidence for improving the SEB outcomes of students within a specific elementary school context. This evidence to practice gap can be addressed by identifying what models and practices work for whom and under what conditions. To achieve this goal, innovations in research synthesis informed by implementation science are needed. The project team will address this gap by using the Distillation and Matching Model to synthesize the literature focused on models and practices designed to improve the SEB outcomes of elementary students.

Project Activities: Developed for the mental health field, the Distillation and Matching Model is a novel approach to research synthesis that is designed to identify what works for whom and under what conditions. This application of the Distillation and Matching Model to the elementary SEB literature will involve three steps. The first step, distillation, involves the identification of practice elements, defined as the goal or general principle that guides a discrete practice (e.g., praise) targeting a specific domain of SEB outcomes (i.e., social, emotional, and behavioral). The researchers will conduct a comprehensive literature review and then distill practice elements designed to target SEB outcomes of elementary students from group design and single case design studies that meet standards of methodological rigor. The distillation process will produce a common set of terms for the individual practices used across the elementary literature that are both studied in isolation and the individual practices that comprise comprehensive intervention models (i.e., manualized programs that include multiple discrete practices). The second step involves the identification of common elements –practice elements found in studies that meet standards of methodological rigor and report significant improvements in student SEB outcomes. Using meta-analytic methods, they will calculate effect sizes at the study level across the two designs (group, single case) and use vote-counting procedures to identify common elements. The third step, matching, is a method for matching common element profiles (combinations of common elements) to intervention (universal, indicated, tertiary) and student characteristics (problem type, age, gender, race/ethnicity) to identify what combinations of common elements work for whom and under what conditions.

Products: The application of the Distillation and Matching Model to the elementary literature represents the next step in a program of research designed to produce actionable information about common elements that researchers and stakeholders (administrators, teachers) can use to develop and evaluate tailored interventions. Accomplishment of the study aims will lead to an IES Development and Innovation project focused on the development of a decision support and implementation toolkit to inform the selection and implementation of interventions tailored to meet the unique SEB needs of elementary students within specific authentic school contexts. The researchers will describe the findings of this study in peer-reviewed publications.

Structured Abstract

Setting: The researchers will identify published and unpublished studies that evaluate models or discrete practices designed to improve the social, emotional, and behavioral (SEB) outcomes of elementary-school aged students.

Sample: The studies included in the synthesis will have samples that include elementary-school aged students.

Factors: The research team will examine what models and practices work under what conditions for elementary students who experience SEB problems. They will accomplish this by matching key intervention (i.e., universal, indicated, tertiary) and student (i.e., problem type, age, gender, race/ethnicity) characteristics to common element profiles (i.e., do different intervention or student characteristics match different combinations of common elements). In addition, the team will determine what common element profiles have the strongest evidence for promoting positive SEB outcomes of elementary-aged students of different problem types, ages, gender, and race/ethnicity across different tiers of intervention.

Research Design and Methods: The project progresses through three phases. Phase I will establish the sample of studies. The researchers will conduct a literature search to identify all relevant unpublished and published studies that use group design (randomized clinical trials, quasi-experimental) and single case designs (SCDs) from the elementary literature focused on practices or models designed to improve the SEB outcomes of elementary students. Phase II will involve descriptive coding for conceptual and methodological characteristics (including intervention and student characteristics), extraction of information needed to calculate effect sizes, and distillation of the practice elements from the studies identified in Phase I. Phase III will involve conducting a separate meta-analysis for the group design and single case design studies, use of the effect sizes to identify the common elements, and then matching them to key intervention (universal, indicated, tertiary) and student (problem type, age, gender, race/ethnicity) characteristics.

Control Condition: Due to the nature of this project, there is no control condition.

Key Measures: The distillation of practice elements from models and practices that target SEB outcomes for elementary students will provide a common set of terms for the practices studied within the elementary literature.

Data Analytic Strategy: First, the research team will conduct a meta-analysis of studies that use a group design in order to produce an ES estimate for each SEB outcome domain evaluated in a study. Second, they will conduct a meta-analysis of the studies that use SCDs in order to produce an effect size estimate for each SEB outcome domain evaluated in a study. Third, using the effect size estimates produced as part of each meta-analysis, they propose to use vote-counting procedures to identify the common elements across studies that use group designs and SCDs. Fourth, they will match common element profiles to the intervention and student characteristics.

Related IES Projects: Development and Validation of a Treatment Integrity Measure of Classroom-Based Instructional Interventions in Early Childhood Settings (R305A140487); BEST in CLASS-Elementary: A Preventative Classroom-based Intervention Model (R305A150246)