Project Activities
The partnership conducted a series of three research studies. First, the EIS was successfully validated across elementary, middle, and high school samples. Next, a social validity study was conducted to gather information from school personnel about their use of EIS data, the importance, and acceptability of the comprehensive identification, prevention model, finding that the large majority of school personnel found the model to be acceptable. The final study looked at the association between fidelity to the model and student outcomes over time.
Structured Abstract
Setting
The setting includes six school districts in Boone County, Missouri: Centralia, Columbia, Hallsville, Harrisburg, Southern, Boone, and Sturgeon. These districts constitute rural and suburban school districts.
Sample
The student samples were predominately White (68-73%). The samples also included Black (13-16%), Asian (4-5%), Latinx (4-5%), and other races (6-7%) students. A total of 15,456 students across elementary, middle, and high school participated in the validation studies, and 16,782 in the fidelity study. A total of 209 school personnel in the social validity study.
The partnership project was guided by the theory of developmental behavioral cascades. Developmental behavioral cascades theory posits that symptoms of social, emotional, and behavioral problems can interfere with adaptive functioning across development. Identifying key indicators of risk such as externalizing problems, social skill deficits, peer relation problems, internalizing problems, inattention, and academic incompetence that can help to identify students in need of supports will lead to improved academic, social behavioral, and emotional outcomes for these students. The theory of change indicates that by identifying students and areas in need of universal prevention, providing appropriate evidence-base supports at the appropriate tier of intervention, and using ongoing data collection and data-based decision making, symptoms of academic, social behavioral, and emotional risk will decrease, resulting in increased academic engagement, reductions in behavior problems, increased time on task and for instruction, and ultimately improved academics.
Initial research
This project built on prior work done with the EIS in Boone County by using data gathered with the EIS and the Behavioral and Emotional Screening System to examine the reliability, validity, and sensitivity of the EIS. The project involved three related studies: (1) validating the EIS to summarize county, district, school, grade, and individual student level social, behavioral, emotional, and academic risk factors; (2) examining the social validity of the EIS among teachers completing the assessment and school-based problem-solving teams who use the data to select appropriate evidence-based programs (EBPs) at the universal, selective, and indicated levels; and (3) exploring the relationship between implementation fidelity of the EIS and outcomes for students.
Control condition
Due to the nature of the design, there was no control condition.
Key measures
In addition to researcher-developed measures (e.g., focus group protocols; a measure of perceptions of the social validity of the EIS), the researchers administered the BASC-3 Behavioral and Emotional Screening System teacher and student scales (Reynolds & Kamphaus, 2015)—which assess problem behavior, social problems, and adaptive skills—to validate the EIS. Missouri Performance Assessment state achievement scores, office disciplinary referrals, suspension and expulsion rates, were collected from school archival records each year of the project.
Data analytic strategy
For Study (1), the research team used confirmatory factor analysis and classification accuracy statistics. For Study (2), researchers coded focus group data by themes, and examined descriptive data for items on a social validity measure. For Study (3), the researchers conducted linear growth modeling and growth mixture modeling over a three period to determine changes in symptoms over time, identify developmental trajectories, and used latent class regression analysis to determine relationships between fidelity and student outcomes.
Key outcomes
- The Early Identification System is a valid universal screening measure of social, emotional, behavioral, and academic risk for students in grades Kindergarten to 12th grade (Herman, et al., 2020; Thompson, et al., 2020). The majority of school personnel find the data to be important and the model developed to link these data to evidence-based interventions acceptable. Further, higher fidelity to the model is associated with better outcomes for students over time.
- Findings indicated that overall students reported a decrease in social, emotional, and behavioral symptoms across a three-year period. Further, growth mixture modeling identified four developmental trajectories of student problems, including students with high levels of problems, students with decreasing problems, students with increasing problems, and students with stable, low levels of problems. Youth in the increasing class were significantly more likely to attend a school with lower fidelity. Further, the association between the high stable class and fidelity approached significance (p=.06) in that students in this class were more likely to attend a low fidelity school. Thus, having higher fidelity to the model will likely produce better student outcomes over time (Reinke, et al., 2020).
People and institutions involved
IES program contact(s)
Project contributors
Products and publications
Project website:
Publications:
Herman, K. C., Reinke, W. M., Huang, F. L., Thompson, A. M., & Doyle-Barker, L. (2020). An investigation of the psychometric properties of the early identification system-student report in a middle school sample. School Psychology.
Reinke, W.M., Herman, K. C., Thompson, A., McCall, C., Copeland, C., Holmes, S., & Owens, S. (2020). Investigating the longitudinal association between fidelity to a large-scale comprehensive school mental health prevention and intervention model and student outcomes. School Psychology Review.
Thompson, A. M., Huang, F., Smith, T., Reinke, W. M., & Herman, K. C. (2020). Confirmatory Factor Structure and Predictive Validity of the Early Identification System—Student Report in a Community Sample of High School Students. School Mental Health, 1-13.
Related projects
Supplemental information
Co-Principal Investigators: Tanner-Jones, Lou Ann; Thompson, Aaron; Herman, Keith; Huang, Frances; Owens, Sarah; Ford, Darin; Felmlee, Chris; Downs, John; Stiepleman, Peter; Schultz, Shawn; Proctor, Lynn
Partner Institutions: The Boone County Schools Mental Health Coalition (BCSMHC)is a collaboration of six independent school districts in Boone County, Missouri and researchers at the University of Missouri's College of Education, Department of Educational School and Counseling Psychology and School Counseling, the Missouri Prevention Center, and the School of Social Work.
Questions about this project?
To answer additional questions about this project or provide feedback, please contact the program officer.
