Description:

Purpose: The purpose of this measurement project was to further develop and validate the Virtual Environment for Social Information Processing (VESIP), a computer-based assessment of social information processing skills in third through seventh grade students. VESIP uses an interactive, simulated environment with customizable avatars to assess students' understanding of different types of social encounters (e.g., bullying/teasing, joining a group of peers already playing together) typical in school settings (e.g., classroom, cafeteria). This project converted VESIP to a web-based platform, established the reliability and validity of the tool, and established provisional national normative data. Accurate measurement of social information processing and conflict resolution is relevant to education because these skills are strong predictors of academic success and peer relationships in school-age children. With respect to practice, VESIP is intended to provide schools with a reliable tool for the accurate and easy measurement of students' social information processing skills, providing necessary information for educators to identify students who may need more support.

Project Activities: The interdisciplinary team developed a web-based delivery platform including an administrative user interface that allows for individual student registration in the system, classroom roster uploads, launching the assessment, and automated score report generation. In this project, we assessed the psychometric properties of VESIP and established preliminary normative data including data across several states.

Key Outcomes:

• VESIP demonstrated good reliability and construct-related validity as a measure of social information processing. Confirmatory factor analyses supported and extended models of social information processing such that it included five factors, one for each distinct domain, and the addition of an overall composite factor and situation type factors improved model fit. Teacher-report of social competence and peer acceptance were also associated with higher scores on VESIP.
• While no direct relationships were identified between AIMSweb and VESIP, significant relationships between VESIP performance and teacher-report of academic competence on the SSIS were identified.
• Finally, test-retest data supported no significant difference in response patterns and highly correlated scores. VESIP is a reasonable assessment solution for educators interested to measure students' social information processing skills.

Structured Abstract

Setting: The validation field trial took place in schools in three suburban Illinois school districts. The large-scale norming study was open to students nationwide and ultimately enrolled students from 10 districts across six states (IL, PA, MI, MS, Washington D.C., and NY), representing three of the four national regions.

Sample: Third through seventh grade students were eligible. Three hundred thirty-four local Chicago-area students participated in the validation field trial. Subsequently, 2,156 students participated in the multi-state norming study. These school districts in the norming study offered a diverse population from urban, suburban, and rural areas. The distribution of enrolled students was spread across the five grade levels: 3rd (9.6%), 4th (21.2%), 5th (28.7%), 6th (19.6%), and 7th (21.1%). The resulting demographic for both the norming and validation samples included a majority of Hispanic participants (54% and 61%, respectively), with White being the second largest demographic (21% and 29%, respectively).

Assessment: Social information processing skills are crucial for interacting successfully with peers and for creating an inclusive school culture and climate where students have the necessary competencies for navigating challenging social situations with classmates. This assessment provides teachers with useful information about what may be a comprehension challenge versus an execution challenge – at both the individual and classroom level. When incorporating social-emotional curricula, it this assessment can provide useful information for benchmarking student level and understanding areas of strength and areas for growth.

Virtual Environment for Social Information Processing (VESIP) was originally developed under a Phase I/II Small Business Innovation Research (SBIR) grants from the National Institute on Disability and Rehabilitation Research. VESIP is based on the Crick and Dodge six-step model of social information processing (solution preference, problem identification, emotion response, intent attribution, goal preference, and social self-efficacy). With IES funding, VESIP was converted into a web-based tool with a broad range of personalization features for a more ecologically-valid assessment experience. VESIP also has an extensive user dashboard for creating classroom rosters, downloading score reports, and launching the assessment.

Research Design and Methods: The project took place in three phases. Phase I focused on building out a web-based delivery platform to support wide-scale administration and development of an automated score report for ease of use by school administrators. In Phase II, to assess the psychometric properties of VESIP, students completed VESIP, an alternate social information processing assessment, and academic measures (see below). In addition, teachers and other school staff familiar with validation students completed social competence questionnaires. During Phase III, VESIP was administered to a sample of students recruited across six states and three regions. These data served as the preliminary normative data for comparison. Finally, a subsample of 63 third through seventh grade students also completed VESIP at a second time point, separated by approximately two weeks, to assess test-retest stability.

Key Measures: For the field trials, VESIP was the main measure of interest. For validation purposes, researchers administered standardized and commercially-available measures, the Social Information Processing Application, School Social Behavior Scales - Second Edition, and the research-based short-form of the teacher-rated Taxonomy of Problematic Social Situations for Children. In addition, the relationship of social information processing to academic achievement was assessed using the standardized AIMSweb reading and math tests. A brief, two-subtest form of the Weschler Abbreviated Scale of Intelligence, Second Edition was also administered to all validation study participants. Further, a subset of students in the validation study had independently completed the Peer Nominations sociometric module of SELweb Early Elementary and those data were shared for consented participants. Separately, deidentified teacher-reported social skills and problem behaviors were collected via the Social Skills Improvement System for a subsequent add-on study of a sixth and seventh grade students who completed VESIP.

Data Analytic Strategy: Internal consistency reliability was assessed via Cronbach's a. Researchers used Rasch modeling to evaluate aspects of construct validity for each dimension of social information processing that is measured by VESIP, to estimate item and threshold difficulties, to screen items for adherence to model requirements, and to assess differential item functioning. Researchers also used multi-level path analyses to evaluate the convergent and discriminant validity of the assessment. Finally, the research team used hierarchical linear modeling and multi-level structural equation modeling to evaluate the relationship between social information processing scores and criterion measures.

Related Projects: Validation of a Spanish-Language Social Reasoning Assessment for Spanish-Speaking English Language Learners (R305A200463)

Products and Publications

ERIC Citations:Find available citations in ERIC for this award here.

Select Publications

Russo-Ponsaran, N., McKown, C., Karls, A., & Wu, I. (2021). Psychometric properties of Virtual Environment for Social Information Processing (VESIP), a social information processing simulation assessment for children. Social Development 30(3), 615–640. https://doi.org/10.1111/sode.12512.

Additional Online Resources and Information:

VESIP page on RNBC site: http://rnbc.org/research/vesip/