Grant Closed

Web-Based Assessment of Social-Emotional Comprehension and Execution in Grades Four to Six

NCER
Program: Education Research Grants
Program topic(s): Social, Emotional, and Behavioral Context for Teaching and Learning
Award amount: $1,399,997
Principal investigator: Clark McKown
Awardee:
Rush University Medical Center
Year: 2016
Project type:
Measurement
Award number: R305A160053

Purpose

Building on prior IES-funded work, the purpose of this measurement project was to develop SELweb LE (Late Elementary), a suite of scientifically sound, usable, and feasible tools for assessing critical dimensions of social-emotional skill in all children in grades 4 to 6. Given that children's social-emotional skill affects their participation and success in school and life, a growing number of states are adopting learning standards that indicate what social-emotional skills children should know or be able to do at different ages. At the same time, there are few systems for directly assessing social-emotional skill that can be used to guide educational practice. SELweb LE measures constructs that correspond to: (a) state social-emotional learning standards, (b) prominent models of social-emotional learning, and (c) factors known to be related to social and academic functioning.

Project Activities

The researchers developed and evaluated SELweb LE in three phases. The initial phase involved item development and tryouts along with field testing to determine SELweb LE's factor structure, score reliability, and convergent, discriminant, and criterion-related validity. In the second phase, they replicated the initial field trial with a larger sample of children and tested measurement equivalence across ethnicity, sex, and language (English and Spanish). In Phase III, they expanded the item pool and revised the number of items per subtest and again re-evaluated SELweb LE's factor structure, score reliability, and criterion-related validity.

Structured Abstract

Setting

Across three phases, this project was conducted in 124 schools in 25 school districts throughout the United States.

Sample

Participants included students in grades 4 to 6. A total of 15,097 students participated.
Assessment
SELweb LE is deployed in schools via a web-based assessment delivery platform and includes direct assessments of children's social-emotional competence. Social-emotional competence subtests measure complex emotion recognition, social perspective-taking, social problem-solving, emotion regulation knowledge, and self-reported self-control.

Control condition

There is no control condition for this project.

Key measures

During field trials, the key measure was SELweb LE. In addition, in some of the field trials, the researchers collected or obtained additional assessments for the purposes of developing validity evidence. In one field trial, for example, the team administered an alternate measure of social and emotional competence and teacher rating scales that assessed student social skills, problem behavior, and academic competence. In another field trial, the team collected teacher rating scale data on students' overall social and emotional competence.

Data analytic strategy

Across three independent samples, the researchers evaluated score reliability, factor structure, convergent and discriminant validity, criterion-related validity, and measurement equivalence across sex, ethnicity, and language form (English and Spanish).

People and institutions involved

IES program contact(s)

Emily Doolittle

Team Lead for Social Behavioral Research
NCER

Products and publications

ERIC Citations: Find available citations in ERIC for this award here.

Journal article, monograph, or newsletter

McKown, C., Russo-Ponsaran, N., & Karls, A. (2022) Web-based assessment of social and emotional competence in the late elementary grades. Social Development, 32(1): 73-97.

McKown, C. (2019). Challenges and Opportunities in the Applied Assessment of Student Social and Emotional Learning. Educational Psychologist, 54 (3): 205–221.

Books and Book Chapters

Russo-Ponsaran, N.M., Karls, A., & McKown, C. (2022). Novel technology based SEL assessments. In J. Burrus, S.H. Rikoon & M.W. Brenneman (Eds.) Assessing Competencies for Social and Emotional Learning: Conceptualization, Development, and Applications. New York: Routledge.

McKown, C. (2022). The fidget spinner effect: Social and emotional assessment and the health evolution of the SEL field. In S.M. Jones, N. Lesaux, & S.P Barnes (Eds.). Measuring and Assessing Non-Cognitive Skills to Improve Teaching and Learning. New York: Guilford Press.

Related projects

A Toolkit for Identifying and Assessing Socially Rejected Children

R305A110143

Web-Based Assessment of Social-Emotional Skills in Middle School

R305A200220

Supplemental information

Key Outcomes: The main features of the assessment and outcomes of the validation study are as follows:

  • SELweb LE (Late Elementary) is a valid and reliable assessment that can be used with children from diverse backgrounds in 4th to 6th grade to measure their understanding of other's thoughts and feelings, their ability to solve social problems, and their ability to engage in self-control (McKown, Russo-Ponsaran, and Karls, 2022).

Research Design: The project took place in three phases. In Phase I, the researchers developed a web delivery platform and an initial item pool for each of the assessment modules. They conducted iterative item tryouts with 15 students in each tryout to evaluate administration procedures, usability, and feasibility. In an initial field trial, they evaluated SELweb LE's factor structure, score reliability, and convergent, discriminant, and criterion-related validity in a sample of 1,011 children. In Phase II, they conducted a second field trial with a larger sample of 10,818 children and tested measurement equivalence across ethnicity, sex, and language. Based on the results of the two field trials they created additional items and alternate forms. In Phase III, they evaluated the expanded item pool and revised the number of items per subtest and determined the factor structure, score reliability, and criterion-related validity using additional validation measures in a sample of 3,268 children.

Website: https://xsel-labs.com/assessments/selweb/

Questions about this project?

To answer additional questions about this project or provide feedback, please contact the program officer.

 

Questions about this project?

To answer additional questions about this project or provide feedback, please contact the program officer.