Project Activities
The study will leverage the experimental design from the original cluster randomized trial of INSIGHTS (22 schools) and also access administrative follow-up data through the end of high school from the NYC Department of Education (provided via the Research Alliance for NYC Schools) on students enrolled in schools at the time of random assignment. There were 1,643 students originally enrolled in the trial, the large majority of whom were eligible for free or reduced price lunch. The research team expects to access follow-up information on 1000 students from the original study, with similar levels of attrition by group. They will fit a series of regressions with blocking group fixed effects to estimate intent-to-treat effects on academic outcomes in middle school (standardized test scores, grades) and high school (9th grade on-track indicator, Regents exams passed, grades, high school graduation, enrollment in college). They will consider variation in impacts by child socioeconomic status, temperament, baseline student skills, and school climate. They will explore mediators (e.g., social-emotional skills, behaviors, and relationships with teachers and peers) and moderators (e.g., program dosage, subsequent school climate, time spent in high school during the coronavirus pandemic) of treatment impacts.
Structured Abstract
Setting
The original cluster randomized trial took place in New York City from 2008 to 2012. The team recruited 22 elementary schools serving primarily students from low-income families located in neighborhoods with high levels of poverty.
Sample
The original study sample included 1,643 students and the team expects to access data through high school on 61.5% (1,011) of these students, with similar retention across conditions. The study team enrolled three successive cohorts of schools from 2008 to 2010. The intervention took place during students' kindergarten and first grade years. Students in the sample were predominantly Black (74%) and eligible for free or reduced-price lunch (86%).
INSIGHTS is a comprehensive, universal SEL intervention that includes teacher, parent, and child programs. Guided by temperament theory, the intervention aims to modify children's learning environments to be sensitive and responsive to their temperaments. Teachers, parents, and children each participated in 10 weeks of intervention with a trained facilitator during both the kindergarten and first grade year. There were high levels of fidelity and dosage.
Research design and methods
The proposed study will leverage the original randomized control trial in order to estimate causal effects of INSIGHTS on long-term outcomes. After agreeing to participate in the study, schools in the original trial were randomly assigned to receive the INSIGHTS intervention across two years or to an attention-control condition. Following program implementation, the team collected follow-up data on students and found short-term intervention impacts on literacy, math, social-emotional, and behavioral skills. A medium-term follow-up study revealed sustained benefits of INSIGHTS on students' language and literacy skills and attendance, as well as reductions in special education receipt through sixth grade.
Control condition
Schools not receiving INSIGHTS were randomly assigned to receive a supplementary reading program to support literacy. The program was offered to children for 10 weeks per year after school and to parents and teachers for two weeks per year. There were high levels of fidelity to the Read Aloud program model.
Key measures
In this long-term follow-up study the team will access administrative data on students in middle school, high school, and beyond. Academic outcomes of interest will be English/Language Arts and Mathematics standardized test scores in 7th and 8th grade, grades and GPA in middle and high school, a 9th grade on-track indicator, Regents exams passed, high school graduation, and college enrollment. In addition, the team will leverage data collected from the original trial and the medium-term follow-up study to assess mediators (social-emotional skills, behaviors, relationships with teachers and peers, receipt of special education, grade retention) and moderators (program dosage, subsequent school climate, time spent in high school during the coronavirus pandemic) of treatment impacts.
Data analytic strategy
The team will fit regressions with blocking group fixed effects, adjusting for baseline covariates, to estimate intent-to-treat impacts on academic outcomes in 7th and 8th grade (standardized test scores, grades) and high school (on-track in 9th grade, Regents exams passed, grades, high school graduation, college enrollment). They will add interactions to the models to examine impact variation by child socioeconomic status, temperament, and baseline skills and school climate. They will use instrumental variables estimation and principal score matching to examine post-random assignment mediators and moderators of treatment impacts.
Cost analysis strategy
The team will use the ingredients method to conduct a cost analysis.
People and institutions involved
IES program contact(s)
Project contributors
Products and publications
The research team will report findings in peer-reviewed journal articles, policy briefs with clear cover pages summarizing results, Op-eds, a podcast, targeted briefings for the NYCDOE and key stakeholders, social media activity via Twitter and Instagram, and conferences of academics and practitioners. Findings will highlight whether early investments in SEL yield long-term benefits for students, identify the key short- and medium-term outcomes that link to impacts through high school and beyond, and provide information on how to target SEL investments toward the students and schools most in need.
Related projects
Questions about this project?
To answer additional questions about this project or provide feedback, please contact the program officer.
