Social and Behavioral Context for Academic Learning
A Toolkit for Identifying and Assessing Socially Rejected Children
Purpose: In this project, researchers developed a suite of scientifically sound, usable tools for screening social rejection and assessing social and emotional learning (SEL) in children in grades K–3 called SELweb. Elementary school-aged children who are rejected by their peers are at elevated risk for academic, behavioral, and emotional problems, yet few tools are available for school professionals to identify socially rejected children and to pinpoint SEL deficits that contribute to their rejection. Behavior rating scales provide important information but are not well-suited to assessing cognitive, affective, and regulatory processes. Direct assessments have technical limitations or are difficult to administer, score, and interpret, which prevents their productive use in schools.
Project Activities: This study was conducted in three phases. In phase 1, the researchers developed item pools for the assessment of four SEL skills along with a web-based software platform. The initial items were used to evaluate the feasibility of the administration procedures. During phase 2, the researchers evaluated the convergent, divergent, and construct validity of SELweb compared to other direct assessments of SEL skills and parent and teacher ratings. In phase 3, the researchers recruited additional children from other states for further validation and preliminary norming.
Key Outcomes: The main features of the assessment and outcomes of the validation study are as follows:
Setting: This project was conducted in elementary schools in Illinois, Texas, California, New York, Connecticut, and Florida.
Population: Over 4000 children in grades K to 3 participated across all 3 phases.
Measure: SELweb is a web-based toolkit of SEL assessments that can be completed within 30 minutes. The toolkit can be used flexibly and in conjunction with existing assessments to conduct efficient universal two-stage screening for social rejection and diagnostic assessment of factors that contribute to social rejection. This information can then be used to inform individualized intervention planning.
SELweb assessments are meant to be used in conjunction with, not in place of, other well-established instruments that are well-suited to assessing children's social behavior. SELweb includes assessments of four SEL skills related to social functioning: nonverbal awareness, “mind sharing” (perspective taking), social problem-solving, and self-regulation. The nonverbal awareness task evaluates children's ability to identify others' emotions from photographs of faces varying in intensity of facial expressions. A second assessment (the pictorial theory of mind task) is intended to reduce language demands of typical theory of mind tasks and enable a more accurate evaluation of children's ability to infer others' mental states (mind sharing). A third assessment evaluates children's ability to engage in effective social problem-solving. A final set of assessments evaluates children's ability to self-regulate.
Research Design and Methods: In phase 1, the researchers developed an item pool for each SEL skill and conducted an initial item tryout to evaluate administration procedures. In parallel, they also developed the web-based software platform for delivery of the assessments. For phase 2, SELweb was co-administered to children along with other direct assessments of SEL skills and parent and teacher ratings of social competence to ascertain reliability and validity. Relying on the analytic findings from phase 1 and 2, a panel of experts on children's social development and assessment helped develop standards of impairment so that SEL assessments can be used for secondary screening of social rejection. The researchers used phase 3 data to confirm the psychometric properties of the SELweb and guide development of preliminary norms.
Key Measures: The researchers used peer- and teacher-report social rejection screeners, "last generation" SEL tests, and the Social Skills Improvement System (SSIS) to validate SELweb.
Data Analytic Strategy: The researchers used Rasch modeling to evaluate validity, estimate item difficulties, and screen items. They used confirmatory factor analysis and structural equation modeling to evaluate convergent, discriminant, and construct validity. The project team used matched moderated regression to conduct differential item functioning. They assessed test-retest reliability in a subset of children during phases 1 and 2.
Related IES Projects: Web-Based Assessment of Social-Emotional Comprehension and Execution in Grades Four to Six (R305A160053), Web-Based Assessment of Social-Emotional Skills in Middle School (R305A200220), SEL Question Digital Platform to Modernize and Advance Assessment of Student Social Emotional Competences (91990021C0028), SEL Quest Digital Platform to Modernize and Advance Assessment of Student Social and Emotional Competencies (91990022C0040),
ERIC Citations: Find available citations in ERIC for this award here.
Dunya, B.A., McKown, C., & Smith, E. (2019). Psychometric Properties and Differential Item Functioning of a Web-Based Assessment of Children’s Emotion Recognition Skill. Journal of Psychoeducational Assessment, 38(5): 627-641.
McKown, C. (2017). Promises and Perils of Assessing Character and Social and Emotional Learning. Journal of Research in Character Education, 13(2): 47–52.
McKown, C. (2017). Social-Emotional Assessment, Performance, and Standards. The Future of Children: Special Issue on Social-Emotional Learning, 27(1): 157-178.
McKown, C. (2018). Reliability, Factor Structure, and Measurement Invariance of a Web-Based Assessment of Children's Social-Emotional Comprehension. Journal of Psychoeducational Assessment, 37(4): 435-449.
McKown, C., Allen, A.M., & Russo-Ponsaran, N.M. (2013). Direct Assessment of Children’s Social-Emotional Comprehension. Psychological Assessment, 25(4): 1154-1166.
McKown, C., Russo-Ponsaran, N.M., Allen, A., Johnson, J.K., and Warren-Khot, H.K. (2016). Social–Emotional Factors and Academic Outcomes Among Elementary-Aged Children. Infant and Child Development, 25(2): 119–136.
McKown, C., Russo-Ponsaran, N.M., Johnson, J.K., Russo, J., and Allen, A. (2016). Web-Based Assessment of Children's Social-Emotional Comprehension. Journal of Psychoeducational Assessment, 34(4): 322–338.
Russo, J.M., McKown, C., Russo-Ponsaran, N.M., and Allen, A. (2017). Reliability and Validity of a Spanish Language Assessment of Children's Social-Emotional Learning Skills. Psychological Assessment, 30(3): 416–421.
Russo-Ponsaran, N.M., Lerner, M.D., McKown, C., Weber, R. J., Karls, A., Kang, E. & Sommer, S.L. (2019). Web-Based Assessment of Social-Emotional Skills in School-Aged Youth with Autism Spectrum Disorder. Autism Research, 12: 1260-1271.