Co-Principal Investigators: O'Connor, Erin; Kemple, James

Purpose: The aim of this study is to conduct a long-term efficacy follow-up of INSIGHTS into Children's Temperament. INSIGHTS is a comprehensive school based preventive intervention that aims to support children's social-emotional development in order to improve their behaviors, social-emotional well-being, and academic skills during the transition to elementary school. Findings from the original cluster randomized trial of INSIGHTS implemented in kindergarten and first grade in schools serving students from low-income families (2008 to 2012) revealed positive short-term treatment impacts on children's literacy, math, sustained attention, and behavioral skills. Follow-up data demonstrated sustained impacts on target outcomes through sixth grade. Although these medium-term impacts are promising, questions remain about whether INSIGHTS continues to benefit children as they move through adolescence, as well as the mechanisms through which the program generates long-term outcomes, the students who benefit the most from the intervention, and the implementation and school-level factors that are critical to promoting sustained impacts. This study will estimate long-term effects of INSIGHTS on students' academic, social-emotional, and behavioral outcomes from 7th to 12th grade and during the transition to college. The study will explore how INSIGHTS promoted long-term outcomes, for whom, and under what circumstances. Findings will provide information on whether early investments in SEL yield benefits for students through high school and beyond

Project Activities: The study will leverage the experimental design from the original cluster randomized trial of INSIGHTS (22 schools) and also access administrative follow-up data through the end of high school from the NYC Department of Education (provided via the Research Alliance for NYC Schools) on students enrolled in schools at the time of random assignment. There were 1,643 students originally enrolled in the trial, the large majority of whom were eligible for free or reduced price lunch. The research team expects to access follow-up information on 1000 students from the original study, with similar levels of attrition by group. They will fit a series of regressions with blocking group fixed effects to estimate intent-to-treat effects on academic outcomes in middle school (standardized test scores, grades) and high school (9th grade on-track indicator, Regents exams passed, grades, high school graduation, enrollment in college). They will consider variation in impacts by child socioeconomic status, temperament, baseline student skills, and school climate. They will explore mediators (e.g., social-emotional skills, behaviors, and relationships with teachers and peers) and moderators (e.g., program dosage, subsequent school climate, time spent in high school during the coronavirus pandemic) of treatment impacts.

Products: The research team will report findings in peer-reviewed journal articles, policy briefs with clear cover pages summarizing results, Op-eds, a podcast, targeted briefings for the NYCDOE and key stakeholders, social media activity via Twitter and Instagram, and conferences of academics and practitioners. Findings will highlight whether early investments in SEL yield long-term benefits for students, identify the key short- and medium-term outcomes that link to impacts through high school and beyond, and provide information on how to target SEL investments toward the students and schools most in need.

Structured Abstract

Setting: The original cluster randomized trial took place in New York City from 2008 to 2012. The team recruited 22 elementary schools serving primarily students from low-income families located in neighborhoods with high levels of poverty.

Sample: The original study sample included 1,643 students and the team expects to access data through high school on 61.5% (1,011) of these students, with similar retention across conditions. The study team enrolled three successive cohorts of schools from 2008 to 2010. The intervention took place during students' kindergarten and first grade years. Students in the sample were predominantly Black (74%) and eligible for free or reduced price lunch (86%).

Intervention: INSIGHTS is a comprehensive, universal SEL intervention that includes teacher, parent, and child programs. Guided by temperament theory, the intervention aims to modify children's learning environments to be sensitive and responsive to their temperaments. Teachers, parents, and children each participated in 10 weeks of intervention with a trained facilitator during both the kindergarten and first grade year. There were high levels of fidelity and dosage.

Research Design and Methods: The proposed study will leverage the original randomized control trial in order to estimate causal effects of INSIGHTS on long-term outcomes. After agreeing to participate in the study, schools in the original trial were randomly assigned to receive the INSIGHTS intervention across two years or to an attention-control condition. Following program implementation, the team collected follow-up data on students and found short-term intervention impacts on literacy, math, social-emotional, and behavioral skills. A medium-term follow-up study revealed sustained benefits of INSIGHTS on students' language and literacy skills and attendance, as well as reductions in special education receipt through sixth grade.

Control Condition: Schools not receiving INSIGHTS were randomly assigned to receive a supplementary reading program to support literacy. The program was offered to children for 10 weeks per year after school and to parents and teachers for two weeks per year. There were high levels of fidelity to the Read Aloud program model.

Key Measures: In this long-term follow-up study the team will access administrative data on students in middle school, high school, and beyond. Academic outcomes of interest will be English/Language Arts and Mathematics standardized test scores in 7th and 8th grade, grades and GPA in middle and high school, a 9th grade on-track indicator, Regents exams passed, high school graduation, and college enrollment. In addition, the team will leverage data collected from the original trial and the medium-term follow-up study to assess mediators (social-emotional skills, behaviors, relationships with teachers and peers, receipt of special education, grade retention) and moderators (program dosage, subsequent school climate, time spent in high school during the coronavirus pandemic) of treatment impacts.

Data Analytic Strategy: The team will fit regressions with blocking group fixed effects, adjusting for baseline covariates, to estimate intent-to-treat impacts on academic outcomes in 7th and 8th grade (standardized test scores, grades) and high school (on-track in 9th grade, Regents exams passed, grades, high school graduation, college enrollment). They will add interactions to the models to examine impact variation by child socioeconomic status, temperament, and baseline skills and school climate. They will use instrumental variables estimation and principal score matching to examine post-random assignment mediators and moderators of treatment impacts.

Cost Analysis. The team will use the ingredients method to conduct a cost analysis.

Related IES Projects: Testing the Efficacy of INSIGHTS in Enhancing the Academic Learning Context(R305A080512); Does Early Intervention Benefit Social-Emotional and Academic Development in Middle School? A Follow-up Study of INSIGHTS (R305A160177); Testing the Efficacy of INSIGHTS for Promoting Positive Learning Environments and Academic Achievement in Nebraska: A Replication Study (R305A180290); New INSIGHTS: A Technology-Enhanced Social and Emotional Learning Intervention (91990022C0037)