Project Activities
This study was conducted in three phases. In phase 1, the researchers developed item pools for the assessment of four SEL skills along with a web-based software platform. The initial items were used to evaluate the feasibility of the administration procedures. During phase 2, the researchers evaluated the convergent, divergent, and construct validity of SELweb compared to other direct assessments of SEL skills and parent and teacher ratings. In phase 3, the researchers recruited additional children from other states for further validation and preliminary norming.
Structured Abstract
Setting
This project was conducted in elementary schools in Illinois, Texas, California, New York, Connecticut, and Florida.
Sample
Over 4000 children in grades K to 3 participated across all 3 phases.
Research design and methods
In phase 1, the researchers developed an item pool for each SEL skill and conducted an initial item tryout to evaluate administration procedures. In parallel, they also developed the web-based software platform for delivery of the assessments. For phase 2, SELweb was co-administered to children along with other direct assessments of SEL skills and parent and teacher ratings of social competence to ascertain reliability and validity. Relying on the analytic findings from phase 1 and 2, a panel of experts on children's social development and assessment helped develop standards of impairment so that SEL assessments can be used for secondary screening of social rejection. The researchers used phase 3 data to confirm the psychometric properties of the SELweb and guide development of preliminary norms.
Key measures
The researchers used peer- and teacher-report social rejection screeners, "last generation" SEL tests, and the Social Skills Improvement System (SSIS) to validate SELweb.
Measure: SELweb is a web-based toolkit of SEL assessments that can be completed within 30 minutes. The toolkit can be used flexibly and in conjunction with existing assessments to conduct efficient universal two-stage screening for social rejection and diagnostic assessment of factors that contribute to social rejection. This information can then be used to inform individualized intervention planning.
SELweb includes
- a universal screener to identify children who are socially rejected
- a set of assessments that are easy to administer, score, and interpret that efficiently diagnose SEL processing deficits for children who screen positive for social rejection
- a set of assessments that lend themselves to the development of educational treatment and service plans to address each child's specific needs
SELweb assessments are meant to be used in conjunction with, not in place of, other well-established instruments that are well-suited to assessing children's social behavior. SELweb includes assessments of four SEL skills related to social functioning: nonverbal awareness, “mind sharing” (perspective taking), social problem-solving, and self-regulation. The nonverbal awareness task evaluates children's ability to identify others' emotions from photographs of faces varying in intensity of facial expressions. A second assessment (the pictorial theory of mind task) is intended to reduce language demands of typical theory of mind tasks and enable a more accurate evaluation of children's ability to infer others' mental states (mind sharing). A third assessment evaluates children's ability to engage in effective social problem-solving. A final set of assessments evaluates children's ability to self-regulate.
Data analytic strategy
The researchers used Rasch modeling to evaluate validity, estimate item difficulties, and screen items. They used confirmatory factor analysis and structural equation modeling to evaluate convergent, discriminant, and construct validity. The project team used matched moderated regression to conduct differential item functioning. They assessed test-retest reliability in a subset of children during phases 1 and 2.
Key outcomes
Key Outcomes: The main features of the assessment and outcomes of the validation study are as follows:
- SELweb's modules reflect a four-factor model of social and emotional learning consisting of emotion recognition, social perspective taking, social problem solving, and self-control (McKown, 2019; McKown et al., 2016).
- SELweb is reliable and valid for ethnically and socioeconomically diverse children in grades K to 3 (McKown, 2019; McKown et al., 2016; Dunya, McKown, and Smith, 2019).
People and institutions involved
IES program contact(s)
Products and publications
ERIC Citations: Find available citations in ERIC for this award here.
Select Publications:
Journal articles
Dunya, B.A., McKown, C., & Smith, E. (2019). Psychometric Properties and Differential Item Functioning of a Web-Based Assessment of Children's Emotion Recognition Skill. Journal of Psychoeducational Assessment, 38(5): 627-641.
McKown, C. (2017). Promises and Perils of Assessing Character and Social and Emotional Learning. Journal of Research in Character Education, 13(2): 47-52.
McKown, C. (2017). Social-Emotional Assessment, Performance, and Standards. The Future of Children: Special Issue on Social-Emotional Learning, 27(1): 157-178.
McKown, C. (2018). Reliability, Factor Structure, and Measurement Invariance of a Web-Based Assessment of Children's Social-Emotional Comprehension. Journal of Psychoeducational Assessment, 37(4): 435-449.
McKown, C., Allen, A.M., & Russo-Ponsaran, N.M. (2013). Direct Assessment of Children's Social-Emotional Comprehension. Psychological Assessment, 25(4): 1154-1166.
McKown, C., Russo-Ponsaran, N.M., Allen, A., Johnson, J.K., and Warren-Khot, H.K. (2016). Social-Emotional Factors and Academic Outcomes Among Elementary-Aged Children. Infant and Child Development, 25(2): 119-136.
McKown, C., Russo-Ponsaran, N.M., Johnson, J.K., Russo, J., and Allen, A. (2016). Web-Based Assessment of Children's Social-Emotional Comprehension. Journal of Psychoeducational Assessment, 34(4): 322-338.
Russo, J.M., McKown, C., Russo-Ponsaran, N.M., and Allen, A. (2017). Reliability and Validity of a Spanish Language Assessment of Children's Social-Emotional Learning Skills. Psychological Assessment, 30(3): 416-421.
Russo-Ponsaran, N.M., Lerner, M.D., McKown, C., Weber, R. J., Karls, A., Kang, E. & Sommer, S.L. (2019). Web-Based Assessment of Social-Emotional Skills in School-Aged Youth with Autism Spectrum Disorder. Autism Research, 12: 1260-1271.
Related projects
Questions about this project?
To answer additional questions about this project or provide feedback, please contact the program officer.
