Project Activities
Researchers will complete a three-phase work plan. In Phase I, researchers will iteratively develop the SELweb MS assessment items. In Phase II, they will conduct two field trials to ascertain the psychometric properties of SELweb MS. In Phase III, researchers will conduct a nationwide norming study.
Structured Abstract
Setting
This project will take place in ethnically and socioeconomically diverse school systems in the Chicago area (Phases I and II) and in school districts nationwide, selected to reflect the regional and demographic diversity of the country (Phase III).
Sample
Participants will include ethnically and socioeconomically diverse samples of students in sixth through eighth grades. During Phase 1, approximately 30 students will participate in focus groups and item tryouts. During Phase II, SELweb MS will be given to 1500 students. During Phase III, 1800 students will complete SELweb MS.
Assessment
SELweb MS will be designed to assess the competencies that are described in the Collaborative for Academic Social and Emotional Learning (CASEL) model of SEL and that are reflected in state SEL standards. The assessment will include three assessment methods—factual questions, situational judgment tasks, and self-report questionnaires. The intended use is to provide assessment data that can be used to guide instruction and measure skill acquisition after a period of instruction. In this way, SELweb MS will be designed to serve as a formative assessment and a tool for continuous improvement. Assessment modules will be deployed in schools via a web-based platform.
Research design and methods
During Phase I, the project team will develop assessment items. The team will use an iterative design process in which they create items and obtain feedback from an expert panel and student focus groups. The researchers will also conduct small-sample item tryouts. The project team will identify cost inputs for a Phase II cost analysis. During Phase II, the team will conduct two field trials to evaluate the psychometric properties of SELweb MS. During Phase III, the researchers will conduct a norming study.
Control condition
: Not applicable.
Key measures
In addition to completing SELweb MS, during Phase II, a subset of students will complete alternate assessments of social-emotional competence and achievement and teachers will complete rating scales assessing the frequency of socially skilled and problem behaviors. The researchers will also collect demographic information about students.
Data analytic strategy
The research team will use Rasch to analyze item and scale properties and test for differential item functioning. Confirmatory Factor Analysis (CFA) will clarify the factor structure of the measures and to further examine measurement equivalence. Hierarchical linear models will evaluate convergent, discriminant, and criterion-related validity. For all SELweb MS scores, the team will develop age-based norms, with means set at 100 and standard deviations at 15.
Cost analysis strategy
During Phase II, the team will gather data about cost inputs during the field trials, including through staff time logs, to estimate total cost, variability in cost, and incremental cost of administering SELweb MS. The researchers will use the CostOut application to analyze total and incremental program cost.
People and institutions involved
IES program contact(s)
Products and publications
Products: Upon completion of Phase III, the team will have developed a normed and validated social-emotional assessment system for middle school. Researchers will also produce peer-reviewed publications and information about cost.
Related projects
Questions about this project?
To answer additional questions about this project or provide feedback, please contact the program officer.
