Purpose: The ability to understand and effectively interact with others is a critical determinant of academic, social, and life success. In recognition of this fact, education policy and practice increasingly focus on teaching students the social-emotional learning (SEL) competencies they need to succeed in school and life. For example, a growing number of states have adopted social-emotional learning (SEL) standards. Despite the importance of SEL skills, few assessments are available for educators to assess social and emotional competencies for the middle school years. The purpose of the proposed project is to develop, field test, and norm SELweb MS, a web-based assessment designed for middle school.

Project Activities: Researchers will complete a three-phase work plan. In Phase I, researchers will iteratively develop the SELweb MS assessment items. In Phase II, they will conduct two field trials to ascertain the psychometric properties of SELweb MS. In Phase III, researchers will conduct a nationwide norming study.

Products: Upon completion of Phase III, the team will have developed a normed and validated social-emotional assessment system for middle school. Researchers will also produce peer-reviewed publications and information about cost.

Structured Abstract

Setting: This project will take place in ethnically and socioeconomically diverse school systems in the Chicago area (Phases I and II) and in school districts nationwide, selected to reflect the regional and demographic diversity of the country (Phase III).

Sample: Participants will include ethnically and socioeconomically diverse samples of students in sixth through eighth grades. During Phase 1, approximately 30 students will participate in focus groups and item tryouts. During Phase II, SELweb MS will be given to 1500 students. During Phase III, 1800 students will complete SELweb MS.

Assessment: SELweb MS will be designed to assess the competencies that are described in the Collaborative for Academic Social and Emotional Learning (CASEL) model of SEL and that are reflected in state SEL standards. The assessment will include three assessment methods—factual questions, situational judgment tasks, and self-report questionnaires. The intended use is to provide assessment data that can be used to guide instruction and measure skill acquisition after a period of instruction. In this way, SELweb MS will be designed to serve as a formative assessment and a tool for continuous improvement. Assessment modules will be deployed in schools via a web-based platform.

Research Design and Methods: During Phase I, the project team will develop assessment items. The team will use an iterative design process in which they create items and obtain feedback from an expert panel and student focus groups. The researchers will also conduct small-sample item tryouts. The project team will identify cost inputs for a Phase II cost analysis. During Phase II, the team will conduct two field trials to evaluate the psychometric properties of SELweb MS. During Phase III, the researchers will conduct a norming study.

Control Condition: Not applicable.

Key Measures: In addition to completing SELweb MS, during Phase II, a subset of students will complete alternate assessments of social-emotional competence and achievement and teachers will complete rating scales assessing the frequency of socially skilled and problem behaviors. The researchers will also collect demographic information about students.

Data Analysis: The research team will use Rasch to analyze item and scale properties and test for differential item functioning. Confirmatory Factor Analysis (CFA) will clarify the factor structure of the measures and to further examine measurement equivalence. Hierarchical linear models will evaluate convergent, discriminant, and criterion-related validity. For all SELweb MS scores, the team will develop age-based norms, with means set at 100 and standard deviations at 15.

Cost Analysis: During Phase II, the team will gather data about cost inputs during the field trials, including through staff time logs, to estimate total cost, variability in cost, and incremental cost of administering SELweb MS. The researchers will use the CostOut application to analyze total and incremental program cost.

Related Projects: Validation of a Spanish-language Social Reasoning Assessment for Spanish-Speaking English Language Learners (R305A200463); Web-Based Assessment of Social-Emotional Comprehension and Execution in Grades Four to Six (R305A160053); A Toolkit for Identifying and Assessing Socially Rejected Children (R305A110143)