Co-Principal Investigator: Borman, Geoffrey

Purpose: The importance of social-emotional learning (SEL) has become widely recognized within the research and policy communities. Evidence suggests that SEL competencies are malleable, can be taught through school-based programs, and can have enduring impacts on academic and employment outcomes. This randomized controlled trial (RCT) of Second Step, a widely-used SEL program designed to impact a range of SEL competencies, will inform current and potential users regarding program impacts and will offer policymakers evidence to assess whether and how a comprehensive SEL program can help students achieve desired SEL skills and academic outcomes. The study also will benefit the research community by assessing the mechanisms through which an array of SEL competencies contribute to students' behavioral and academic outcomes.

Project Activities: Over three years, 60 middle schools will be randomly assigned to implement Second Step or continue with typical practices. The researchers will estimate the program's intention-to-treat (ITT) effects on a range of academic, behavioral, and social-psychological measures of well-being; explore how SEL competencies mediate positive academic and behavioral outcomes; and examine student factors and school-contextual effects that may moderate impacts. An implementation study will document the extent to which Second Step is implemented with fidelity in treatment schools and will also shed light on how the program changes the schoolwide SEL environment relative to comparison schools. Finally, a cost study will provide policymakers and practitioners with relevant program cost and cost-effectiveness estimates needed to guide decision making.

Products: The project team will produce peer-reviewed publications and will share findings with practitioners and policymakers via presentations and other intentional forms of web-based and social-media-based content. The study also will produce large-scale validations of surveys and data collection tools that can be used in future studies of schoolwide SEL programs.

Structured Abstract

Setting: This study will take place in 60 middle schools from one large or medium-sized urban district.

Sample: The researchers will collect data from all students across grades 6–8 during each year of the 3-year study. At each grade level the researchers expect to have 160 students (480 per school) and they plan to collect data from approximately 300 students per school. These assumptions would yield annual final analytic samples of 6,000 students per grade and 18,000 students per year. They estimate that 6 advisory teachers/grade (18/school) will implement the program. For the school-wide measures, they will survey all grade 6–8 teachers in the schools.

Intervention: The content and delivery of the Second Step program was revised in 2017. In terms of content, there is a greater focus on promoting positive outcomes rather than preventing negative outcomes. New units include those that focus on Mindsets and Goals; Thoughts, Emotions, and Decisions; and Managing Relationships and Social Conflict. In terms of delivery, there is professional development for teachers, a comprehensive set of web-based materials to facilitate implementation, and an online dashboard that allows principals to monitor teacher implementation. Second Step is implemented in nearly 25% of U.S. schools serving middle grades (6–8), making it one of the most widely used SEL programs.

Research Design and Methods: The researchers will conduct an efficacy trial designed to meet What Works Clearinghouse (WWC) standards without reservations; an implementation study; and a cost-effectiveness study. The project will be carried out across 5 years, beginning with a recruitment year, followed by 3 years for study implementation, and finishing with a year devoted to data analysis and dissemination. During the first year of this project, the researchers will recruit 60 schools and randomize 30 to treatment and 30 to the control condition. Treatment schools will receive a 3-year license to implement Second Step. Each year, during the fall and spring, all students and teachers will complete a survey. For those grade 6 students who begin the study in Year 1, the researchers will track longitudinal outcomes across grades 6–8. The researchers estimate that advisory teachers will implement the program. They will survey all grade 6-8 teachers for the school-wide measures.

Control Condition: Control schools will implement "business-as-usual" SEL instruction. In this study, "business-as-usual" is defined as schools implementing no formal, replicableSEL program. Control schools may be implementing some SEL content/instruction, which the researchers will measure through the treatment service contrast provided by the implementation study. Control schools will be provided the Second Step program, free of charge, after the study is completed.

Key Measures: Baseline and outcome measures come from district/school administrative data and include student grades and standardized test scores as academic outcomes and attendance and disciplinary records as behavioral outcomes. The student-reported social-emotional outcomes include validated measures of a growth mindset, belongingness, perspective-taking, empathy, problem solving, emotion management, self-awareness, and positive student-teacher relationships. Teachers will take the California School Climate Survey, which measures the degree to which they help foster SEL competencies in the school and in their classrooms. The researchers will derive measures of implementation from digital Second Step program usage data; teacher surveys and implementation log data; and classroom observations.

Data Analytic Strategy: The primary data analyses will be conducted using a two-level multilevel model, with students nested within schools and Second Step treatment assignment modeled at the level of the school. These analyses of school-level intention-to-treat effects of Second Step on the SEL, behavioral, and academic outcomes will be of primary interest. The 3-year study will provide supplemental analyses of longitudinal SEL, behavioral, and academic outcomes and treatment impacts across the middle grades, 6–8. To understand treatment-effect heterogeneity, the researchers will perform analyses of moderation across students and schools. At the student level, evidence and theory suggest that SEL programs may have particularly strong benefits for students from marginalized groups, including racial/ethnic minorities, students from low-income families, those who receive special education services, and those who have low baseline social-emotional competencies and school academic and behavioral outcomes. Such cross-level, student-level by school-level treatment assignment interactions will be estimated within a multilevel framework. School-level moderators to explore include baseline school-level aggregate measures of student social-emotional competencies, behavioral referrals, attendance, academic achievement, and pre-intervention measures of the school SEL climate. Finally, multilevel mediation analyses will unpack how proximal treatment effects on students' reported social-emotional wellbeing and proximal schoolwide effects (increased social-emotional supports and an improved SEL climate) mediate the distal student behavioral and academic outcomes.

Cost Analysis: The researchers will estimate typical program costs across a large sample of schools, itemizing both starting and recurring costs. To estimate the cost of Second Step, and comparable social-emotional programming (if any) implemented in control schools, they will employ the ingredients approach to cost analysis. This approach will provide cost estimates for implementing Second Step and will project program costs if reproduced in a different context. Using program documents, a cost-analysis survey, and interviews with relevant stakeholders (e.g., principals, teachers), they will create a database of program costs. After fitting a resource cost model to these data, they will create a series of cost estimates, including total program cost, and costs per school, per student, per year, and the costs by "ingredients" (e.g., personnel, materials and equipment, professional development). Based on these cost estimates, they will calculate Second Step cost-effectiveness estimates as ratios of impacts to costs at the school and student levels. They will calculate cost-effectiveness estimates for allsocial-emotional, behavioral, and academic outcomes that prove to be of statistical and practical importance.