Web-Based Assessment of Social-Emotional Comprehension and Execution in Grades Four to Six
Purpose: Children's social-emotional skill affects their participation and success in school and life. Because of this, a growing number of states are adopting learning standards that indicate what social-emotional skills children should master at different ages. At the same time, there are few systems for directly assessing social-emotional skill to guide educational practice. The purpose of this project is to develop SELweb LE (Social-Emotional Learning web – Late Elementary), a web-based assessment that includes a suite of scientifically sound, usable, and feasible modules for assessing critical dimensions of social-emotional skill in all children in fourth through sixth grades. The constructs measured correspond to: (a) prominent models of social-emotional learning, (b) factors known to be related to academic and other outcomes, and (c) state social-emotional learning standards.
Project Activities: In Phase I, researchers will develop a web delivery platform and an initial item pool for each assessment module. The research team will conduct Iterative item tryouts with 15 students in each tryout to evaluate administration procedures, usability, and feasibility. Experts will provide feedback on the content of the teacher report assessment. In Phase II, researchers will administer the assessment modules to approximately 2,500 children across two field trials. In addition, they will administer parallel assessments of social-emotional and academic outcomes and teacher report of social behavior. In Phase III, the team will administer SELweb LE to a nationally representative sample of 1,600 children in fourth through sixth grades.
Products: The research team will produce a self-administered, scientifically sound and usable web-based set of modules that assess children's social-emotional comprehension and execution. They will design an assessment system for easy deployment and mass administration in schools to: (a) evaluate student and classroom progress towards meeting SEL standards, (b) clarify students' strengths and instructional needs, (c) understand the needs of students who are considered for intensive in-class SEL instruction (Tier 2) or resource academic, social, and emotional support (Tier 3), and (d) decide what skills to teach to which students. Researchers will also produce peer-reviewed publications.
Setting: This project will take place in Chicago-area school districts (Phases I and II) and school districts nationwide (Phase III).
Population: Participants include students in fourth through sixth grades. A total of approximately 4,100 children will participate during all three phases.
Assessment: The research team will deploy assessment modules in schools via a web-based platform and will include direct assessments of children's social-emotional comprehension and teacher report of children's social-emotional execution. Social-emotional comprehension assessment modules measure emotion understanding, perspective-taking, social problem-solving, emotion regulation knowledge, and delay of gratification. A social-emotional execution module measures teacher report of socially skilled and socially aversive behavior.
Research Design and Methods: In Phase I, the research team will develop a web delivery platform and an initial item pool for each assessment module. They will complete iterative item tryouts with 15 students in each tryout in order to evaluate administration procedures, usability, and feasibility. Experts will provide feedback on the content of the teacher report assessment. In Phase II, researchers will administer the assessment modules to approximately 2,500 children across two field trials. They will also administer parallel assessments of social-emotional and academic outcomes and teacher report of social behavior. In Phase III, the team will administer SELweb LE to a nationally representative sample of 1,600 children in fourth through sixth grades.
Key Measures: The team will assess cognitive ability by administering the Wechsler Abbreviated Scale of Intelligence – Second Edition (WASI-II), and academic skill by administering the reading and math tasks from the AIMSweb system. Researchers will administer alternative assessments of social-emotional comprehension, including the UC Davis Set of Emotion Expressions, the "Strange Stories" assessment, vignettes from the Social Information Processing Application (SIP-AP), and the Conners' Continuous Performance Test 3rd Edition (CPT-III). Teachers will complete the Social Skills Improvement System (SSIS) Rating Scale in order to describe students' social-emotional execution.
Data Analysis: Researchers will use Rasch modeling to evaluate aspects of construct validity for each assessment, estimate item and threshold difficulties, screen items for adherence to model requirements, and assess differential item functioning. They will also use multi-level confirmatory factor analysis to evaluate the factor structure and structural equation modeling to evaluate convergent, discriminant, and criterion related validity. The team will collect data in Phase III from 1,600 students nationwide to confirm the psychometric properties of the assessment modules and to guide development of national norms.
Related IES Projects: A Toolkit for Identifying and Assessing Socially Rejected Children (R305A110143)
Project Website: http://www.selweb.com/le/