Skip Navigation
Funding Opportunities | Search Funded Research Grants and Contracts

IES Grant

Title: Refinement and Validation of a Large-Scale Panel Survey of Students' Social-Emotional Learning in Grades 4 Through 12 in the COVID Era
Center: NCER Year: 2023
Principal Investigator: Meyer, Robert Awardee: Education Analytics, Inc.
Program: Social, Emotional, and Behavioral Context for Teaching and Learning      [Program Details]
Award Period: 3 years (07/01/2023 – 06/30/2027) Award Amount: $1,925,622
Type: Measurement Award Number: R305A230402
Description:

Co-Principal Investigators: Wang, Yang Caroline; Bolt, Daniel M.

Purpose: In this project, researchers propose to refine and validate an open-source social emotional learning (SEL) survey instrument developed by California's CORE Districts for students in grades 4 through 12. Specifically, this project has 3 goals:

  1. to refine the CORE SEL survey to support largely underrepresented student subgroups disproportionately affected by COVID-19
  2. to collect validity evidence from the updated SEL measure (including a direct assessment pilot within CORE and a cross-validation study outside of CORE) to support the various uses and interpretations of SEL scores at both the student and school levels
  3. to report SEL outcomes and develop an accompanying suite of resources to facilitate research and applied purposes.

Researchers will address an unmet demand by practitioners who are seeking a validated instrument to measure student social emotional competencies. They will also contribute to the SEL research field by deepening understanding of the overall and disparate impacts of COVID-19 on social emotional skills that are critical to learning acceleration and recovery.

Project Activities: Researchers will use 11 years of data (2014–15 to 2024–25) from the CORE Districts, a consortium of 8 large urban school districts in California collectively serving more than 1 million students, more than a third of whom are English learners, two-thirds Latinx, and three-quarters from economically disadvantaged backgrounds. The project consists of multiple studies related to the measurement, improvement, and use of the CORE SEL survey. The quantitative studies will apply different approaches from classical test theory, item response theory, and other relevant statistical methodology to investigate and validate the CORE SEL measures, specifically for student subgroups disproportionately affected by COVID-19. The qualitative study will use cognitive labs to further cross-validate findings from the quantitative studies.

Products:  In addition to a validated, open-source student SEL survey instrument, this project will produce a de-identified, longitudinal dataset of SEL measures (including raw and scale scores) for use by researchers around the country, technical documentation to accompany the application of those SEL measures in research, benchmark data and accompanying interpretation guidebooks for use by CORE practitioners, nationally normed SEL benchmarks for use by practitioners nationwide, and a suite of tools to support the use of the CORE SEL survey instrument (including guidebooks and infographics for appropriate interpretation and use of the national norms and cost-analysis results; best practices for SEL instrument selection, administration, reporting, and use; mock-ups of data visualizations to compare nationally normed data side-by-side with local SEL data; and a webinar introducing these tools to a national audience of practitioners).

Structured Abstract

Setting: This research will take place in eight large urban districts in California.

Sample: The participants come from CORE Districts, which is a consortium of 8 school districts in California—Fresno, Garden Grove, Long Beach, Los Angeles, Oakland, Sacramento, San Francisco, and Santa Ana—collectively serving more than one million students (about 20 percent of California's student population). The sample will include all students participating in the SEL survey in grades 4 to 12 from school years 2014–15 through 2024–25. For example, the sample from the 2014–15 school year includes 425,607 students from 1,134 schools; 67 percent of these students were Latinx, 35 percent English learners, and 78 percent from economically disadvantaged backgrounds.

Instrument: The original CORE SEL survey was designed to measure four SEL constructs: self-management, growth mindset, self-efficacy, and social awareness. Each construct includes four to nine survey items where students rate themselves on a five-point Likert scale.

Research Design and Methods: Researchers will conduct 10 related studies intended to improve the CORE's SEL survey. The quantitative methodology will apply different approaches from classical test theory, item response theory (IRT), and other relevant statistical methodology (such as multilevel models). These studies aim to investigate the internal functioning of SEL measures (specifically for student groups disproportionately affected by COVID-19), variance decomposition across students and schools, longitudinal SEL development, relationships with external variables including learning acceleration and recovery during the pandemic, and cross-validation in a diverse student sample outside of CORE. The qualitative methodology will use cognitive labs to further triangulate findings from the quantitative study.

Control Condition: This research design does not have a control condition.

Key Measures: Along with the SEL survey measures, this project will include three additional types of measures:

  1. academic measures (e.g., Smarter Balanced mathematics and English language arts assessments)
  2. non-academic measures (e.g., chronic absenteeism, suspension/expulsion rate, culture/climate survey measures, and an alternative SEL assessment, SELweb LE)
  3. student demographics (e.g., gender, race/ethnicity, economic disadvantage status, English learner status, and disability status)

California's CORE Districts collect an extensive list of student characteristics and student test and non-test outcome data through the California Longitudinal Pupil Achievement Data System (CALPADS), test/survey vendors (e.g., SBAC CA Assess and Panorama), and district data sources.

Data Analytic Strategy: The primary data analytic methods of this study include unidimensional and multidimensional IRT models, IRT mixture models, multilevel factor and regression analysis, differential item functioning and measurement invariance analyses, reliability and validity analyses, and cognitive interviews.

Cost Analysis: The research team will conduct a cost-analysis using CostOut application to compute the full cost of implementation compared to business as usual. They will integrate the results into the study's accompanying resources. This information will be particularly valuable for practitioners outside CORE to make decisions about instrument adoption and use.


Back