Project Activities
The researchers will use multiple research methods including an iterative assessment design process to calibrate and refine the MIDAS model with an existing data set and cross validation of the MIDAS framework. They will also determine whether the MIDAS system is feasible for middle schools to use and how much it will cost.
Structured Abstract
Setting
This project will take place at six middle schools across two sites, including one in the Southeast (Florida) and another in the Midwest (Wisconsin).
Sample
Primary research participants will include middle school students at-risk for emotional and behavioral problems, as verified by multiple universal screening tools, and middle school teachers, who will be responsible for completing various assessment tools.
In this project, researchers will develop an online and automated multi-informant rating system, with related decisional processes. The assessments will include teacher and student versions of the Social, Academic, and Behavioral Risk Screener(SAEBRS).
Research design and methods
For this project, the first phase is to calibrate the MIDAS system using a large extant database of schools that have utilized the SAEBRS (N=6,000) and an item response theory-based scoring system across teacher and student SAEBRS forms. Researchers will calibrate the system with pre-test risk factors and existing data from school wide information systems. The second phase will cross-validate using new universal screenings from two teachers (homeroom, subject area), student-self report, extant student data (such as office discipline referrals), and demographic data (such as ethnicity and age). The research team will use a Bayesian approach to estimate individual student probabilities and evaluate the resulting MIDAS score with respect to test fairness/measurement invariance, usability, and accuracy. A third phase will test the feasibility of MIDAS and associated procedures within educational contexts. The team will give schools access to a MIDAS prototype and trained in its use to inform screening decisions. Via a mixed-methods approach, MIDAS end-users (administrators, teachers, and student support personnel such as school psychologists, counselors, and social workers) will then provide information regarding the feasibility of MIDAS implementation at the end of the school year. These data will inform how MIDAS will be used in real-world decision-making, with a focus on usability and sustainability. The research team will use findings from the final phase to inform the refinement of the MIDAS prototype, implementation guidance, and training procedures.
Control condition
Due to the nature of the study, there is no control condition.
Key measures
Measures include SAEBRS and various extant academic data such as test scores and curriculum-based measurements and behavioral data such as office discipline referrals.
Data analytic strategy
The research team will examine MIDAS development, refinement, and validation using modern psychometric analyses including a Bayesian data analytic framework, item response theory, and correlational analyses. Analyses will incorporate the multi-level nature of resulting data and will include hierarchical linear modeling.
Cost analysis strategy
The researchers will determine the cost of MIDAS using the ingredients method. Four categories of basic costs are of interest, including personnel, facilities, equipment and materials, and other costs.
People and institutions involved
IES program contact(s)
Project contributors
Products and publications
This project will yield a fully developed and validated MIDAS computer scoring system, which is intended for use as a multi-informant rating system. This project will also inform the development of resources to support MIDAS implementation including an implementation guide and checklist, prerequisites for MIDAS adoption, and key considerations related to decision-making such as how to interpret the data. A final resource includes an open Application Interface (API) that allow MIDAS to be integrated within existing schoolwide information systems (EduClimber, PowerSchool) to support secure entry and storage of MIDAS data and generation of data reports.
Related projects
Questions about this project?
To answer additional questions about this project or provide feedback, please contact the program officer.
