A Toolkit for Identifying and Assessing Socially Rejected Children
Purpose: During the 2009–2010 school year, approximately 4 million elementary school-aged children were rejected by their peers. These children are at elevated risk for academic, behavioral, and emotional problems. Despite these stark facts, few tools are available for school professionals to identify socially rejected children and to pinpoint social-emotional learning (SEL) deficits that contribute to their rejection. Existing behavior rating scales provide important information, but are not well-suited to assessing cognitive, affective, and regulatory processes. Existing direct assessments have technical limitations or are difficult to administer, score, and interpret. This prevents their productive use in schools. The purpose of this project is to develop a suite of scientifically sound, usable tools for screening social rejection and assessing SEL in children in grades K–3. The web-based toolkit will include assessments that can be completed within 30 minutes. In addition, the toolkit will be used flexibly and in conjunction with existing assessments to conduct efficient universal two-stage screening for social rejection and diagnostic assessment of factors that contribute to social rejection. This information can then be used to inform individualized intervention planning.
Project Activities: Initial activities to be completed in Phase I include the development of item pools for the assessment of four SEL skills, and a web-based software platform. These initial items will be used with 40 children during Phase I to evaluate the feasibility of the administration procedures. During Phase II, researchers will recruit 250 children who will be assessed with the new social rejection screener and SEL assessments, in addition to other direct assessments of SEL skills and parent and teacher ratings. This information will be used to evaluate the convergent, divergent, and construct validity of the assessment. During Phase III, researchers will recruit 500 Illinois schoolchildren who reflect the demographic characteristics of the United States for further validation and for preliminary norming of the screener and SEL assessments. All assessments will be delivered via web-based software with automated scoring and score-reporting.
Products: Products will include a scientifically sound and usable web-based social rejection screener and battery of SEL assessments. Scholarly reports of findings will also be produced.
Setting: This project will be conducted in 4 Chicago-area elementary schools (Phases I and II) and 5 elementary schools throughout Illinois (Phase III).
Population: Participants include eligible students in grades K–3. A total of approximately 800 children will participate during all three phases.
Measure: The proposed assessments will be designed to be deployed in schools via the internet and will include: (a) a universal screener to identify children who are socially rejected; (b) a set of assessments that are easy to administer, score, and interpret that efficiently diagnose SEL processing deficits for children who screen positive for social rejection; and (c) a set of assessments that lend themselves to the development of educational treatment and service plans to address each child's specific needs. These assessments will be designed for use in conjunction with, not in place of, other well-established instruments that are well-suited to assessing children's social behavior. The assessments to be developed in this project include assessments of four SEL skills known to be related to social functioning: nonverbal awareness, mindsharing (perspective taking), social problem-solving, and self-regulation. The nonverbal awareness task will evaluate children's ability to identify others' emotions from photographs of faces varying in intensity of facial expressions. A second assessment, the pictorial theory of mind task, is intended reduce language demands of typical theory of mind tasks, and enable a more accurate evaluation of children's ability to infer others' mental states (mindsharing). A third assessment will evaluate children's ability to engage in effective social problem-solving. A final set of assessments will evaluate children's ability to self-regulate.
Research Design and Methods: In Phase I, an item pool for each SEL assessment will be developed and researchers will conduct an initial item tryout with 40 students to evaluate administration procedures. In parallel, the web-based software platform will be developed for delivery of the assessments. For Phase II, the SEL assessments will be co-administered to 250 children, along with other direct assessments of SEL skills and parent and teacher ratings of social competence. These data will then be analyzed in order to ascertain the reliability and validity of the set of assessments. Relying on the analytic findings from Phase I and Phase II, a panel of experts on children's social development and assessment will help develop standards of impairment so that SEL assessments can be used for secondary screening of social rejection. Phase III data will be collected from 500 students statewide to confirm the psychometric properties of the SEL assessments and guide development of preliminary norms.
Key Measures: Current peer- and teacher-report social rejection screeners, "last generation" SEL tests, and the Social Skills Improvement System (SSIS) will be used to validate the new web-based SEL screener and assessments.
Data Analytic Strategy: Rasch modeling will be used to evaluate each assessment's validity, to estimate item difficulties, and to screen items. Researchers will also use confirmatory factor analysis and structural equation modeling to evaluate convergent, discriminant, and construct validity. The project team will use matched moderated regression to conduct differential item functioning. Test-retest reliability will be assessed in a subset of children during Phases I and II.
Journal article, monograph, or newsletter
McKown, C. (2017). Promises and Perils of Assessing Character and Social and Emotional Learning. Journal of Research in Character Education, 13(2), 47–52.
McKown, C. (2018). Reliability, Factor Structure, and Measurement Invariance of a Web-Based Assessment of Children's Social-Emotional Comprehension. Journal of Psychoeducational Assessment, 0734282917749682.
McKown, C., Russo-Ponsaran, N.M., Allen, A., Johnson, J.K., and Warren-Khot, H.K. (2016). Social–Emotional Factors and Academic Outcomes Among Elementary-Aged Children. Infant and Child Development, 25(2): 119–136.
McKown, C., Russo-Ponsaran, N.M., Johnson, J.K., Russo, J., and Allen, A. (2016). Web-Based Assessment of Children's Social-Emotional Comprehension. Journal of Psychoeducational Assessment, 34(4): 322–338.
Russo, J.M., McKown, C., Russo-Ponsaran, N.M., and Allen, A. (2017). Reliability and Validity of a Spanish Language Assessment of Children's Social-Emotional Learning Skills. Psychological Assessment, 30(3), 416–421.
McKown, C. (2017). Social and Emotional Learning: A Policy Vision for the Future. The Future of Children: Policy Brief.