Project Activities
The researchers will conduct a series of studies to evaluate the psychometric properties and measurement equivalence of VESIP-Sp, as well as develop a language routing approach. This work will include mixed methods to determine things such as cultural appropriateness and will leverage quantitative methods to determine construct, convergent, and discriminant validity of the measure.
Structured Abstract
Setting
This project will be conducted in partner school districts from Illinois and Utah where there are large populations of Spanish-speaking ELL students.
Sample
The team will enroll as many as 1,800 students. Eligible students will include Spanish-speaking ELL students from demographically diverse schools in third to seventh grades.
VESIP-Sp is a trans-adapted Spanish-language version of VESIP, an established web-based direct assessment of students' social reasoning skills (30 to 35 minutes). It uses a simulation format to capture students' skills. Students assume the role of the primary avatar, Alex, who experiences 10 challenging social situations over 2 school days. Alex is queried about his/her understanding of and response to these situations by an avatar named Dani. VESIP includes five types of social situations (such as ambiguous provocation), each presented in two school settings (such as the cafeteria).
Research design and methods
The research team will evaluate the psychometric properties (Aim I) and measurement equivalence (Aim II) of VESIP-Sp (n = 1,150). They will include think aloud protocols and focus groups to determine the cultural appropriateness of VESIP-Sp. Aim III is to establish criterion-related validity (n = 450). ExploratoryAim IV is to develop guidelines for a language routing approach to determine which language-version should be used (n = 200).
Key measures
VESIP-Sp, VESIP- Eng, and one module of SELweb LE-Spanish will measure social reasoning skills through direct assessment. To assess measurement equivalence, the researchers will measure social skills, problem behaviors, and academic competence via the Social Skills Improvement System (teacher and student forms), Academic Competence Evaluation Scales — Short Form, school-provided standardized math and reading scores, and school-provided standardized language proficiency scores. In order to have a good understanding of the characteristics of the Spanish-speaking ELL students participating in the study, schools will be asked to provide data (de-identified or consented) regarding student profiles (e.g., language proficiency level, years in U.S., years in an English-speaking educational system, parent highest academic level, SES status).
Data analytic strategy
Cronbach's a will be used for internal consistency reliability. Rasch modeling will be used to evaluate aspects of construct validity, to estimate item and threshold difficulties, to screen items for adherence to model requirements, and to assess differential item functioning. Multi-level path analyses will evaluate convergent and discriminant validity. Finally, while accounting for the nested structure of the data, hierarchical linear modeling and multi-level structural equation modeling will be conducted to evaluate the relationship between social reasoning scores and criterion measures and between language proficiency and performance.
Cost analysis strategy
The team will use an ingredients-based approach to conduct a cost analysis of the personnel and non-personnel costs associated with VESIP-Sp administration, scoring, and interpretation of data.
People and institutions involved
IES program contact(s)
Products and publications
Products: Expected outcomes include (a) determination of the validity of VESIP-Sp for Spanish-speaking ELL students, (b) guidelines for a language routing approach, (c) an updated user manual in Spanish and English, (d) bilingual publications for lay and scientific audiences, and (e) bilingual presentations at scientific and practice conferences. Results will yield important data relevant to the ELL school and community, the broader field of social-emotional assessment, and policy.
Related projects
Questions about this project?
To answer additional questions about this project or provide feedback, please contact the program officer.
