Description:

Co-Principal Investigators: Tanner-Jones, Lou Ann; Thompson, Aaron; Herman, Keith; Huang, Frances; Owens, Sarah; Ford, Darin; Felmlee, Chris; Downs, John; Stiepleman, Peter; Schultz, Shawn; Proctor, Lynn

Partner Institutions: The Boone County Schools Mental Health Coalition (BCSMHC)is a collaboration of six independent school districts in Boone County, Missouri and researchers at the University of Missouri's College of Education, Department of Educational School and Counseling Psychology and School Counseling, the Missouri Prevention Center, and the School of Social Work.

• validate the Early Identification System (EIS) currently in use by the BCSMHC, a universal assessment system designed to summarize county, district, school, grade, and individual student level social, behavioral, emotional, and academic risk factors;
• examine the social validity of the EIS among teachers completing the assessment and school-based problem-solving teams who use the data to select appropriate evidence-based programs (EBPs) at the universal, selective, and indicated levels; and
• examine the fidelity of the model and the resulting outcomes for students.

Project Activities: The partnership conducted a series of three research studies. First, the EIS was successfully validated across elementary, middle, and high school samples. Next, a social validity study was conducted to gather information from school personnel about their use of EIS data, the importance, and acceptability of the comprehensive identification, prevention model, finding that the large majority of school personnel found the model to be acceptable. The final study looked at the association between fidelity to the model and student outcomes over time.

Key Outcomes: The Early Identification System is a valid universal screening measure of social, emotional, behavioral, and academic risk for students in grades Kindergarten to 12th grade (Herman, et al., 2020; Thompson, et al., 2020). The majority of school personnel find the data to be important and the model developed to link these data to evidence-based interventions acceptable. Further, higher fidelity to the model is associated with better outcomes for students over time.

Findings indicated that overall students reported a decrease in social, emotional, and behavioral symptoms across a three-year period. Further, growth mixture modeling identified four developmental trajectories of student problems, including students with high levels of problems, students with decreasing problems, students with increasing problems, and students with stable, low levels of problems. Youth in the increasing class were significantly more likely to attend a school with lower fidelity. Further, the association between the high stable class and fidelity approached significance (p=.06) in that students in this class were more likely to attend a low fidelity school. Thus, having higher fidelity to the model will likely produce better student outcomes over time (Reinke, et al., 2020).

Products: The partnership summarized the findings in a series of reports and presentations for schools, district, and state partners; national conference presentations; and publications in peer-reviewed journals. The Early Identification System and associated implementation model will be disseminated across the nation.

Structured Abstract

Setting: The setting includes six school districts in Boone County, Missouri: Centralia, Columbia, Hallsville, Harrisburg, Southern, Boone, and Sturgeon. These districts constitute rural and suburban school districts.

Sample: The student samples were predominately White (68–73%). The samples also included Black (13-16%), Asian (4-5%), Latinx (4–5%), and other races (6-7%) students.  A total of 15,456 students across elementary, middle, and high school participated in the validation studies, and 16,782 in the fidelity study.  A total of 209 school personnel in the social validity study.

Malleable Factor: The partnership project was guided by the theory of developmental behavioral cascades. Developmental behavioral cascades theory posits that symptoms of social, emotional, and behavioral problems can interfere with adaptive functioning across development. Identifying key indicators of risk such as externalizing problems, social skill deficits, peer relation problems, internalizing problems, inattention, and academic incompetence that can help to identify students in need of supports will lead to improved academic, social behavioral, and emotional outcomes for these students. The theory of change indicates that by identifying students and areas in need of universal prevention, providing appropriate evidence-base supports at the appropriate tier of intervention, and using ongoing data collection and data-based decision making, symptoms of academic, social behavioral, and emotional risk will decrease, resulting in increased academic engagement, reductions in behavior problems, increased time on task and for instruction, and ultimately improved academics.

Research Design and Methods: This project built on prior work done with the EIS in Boone County by using data gathered with the EIS and the Behavioral and Emotional Screening System to examine the reliability, validity, and sensitivity of the EIS. The project involved three related studies: (1) validating the EIS to summarize county, district, school, grade, and individual student level social, behavioral, emotional, and academic risk factors; (2) examining the social validity of the EIS among teachers completing the assessment and school-based problem-solving teams who use the data to select appropriate evidence-based programs (EBPs) at the universal, selective, and indicated levels; and (3) exploring the relationship between implementation fidelity of the EIS and outcomes for students.

Control Condition: Due to the nature of the design, there was no control condition.

Key Measures: In addition to researcher-developed measures (e.g., focus group protocols; a measure of perceptions of the social validity of the EIS), the researchers administered the BASC-3 Behavioral and Emotional Screening System teacher and student scales (Reynolds & Kamphaus, 2015)—which assess problem behavior, social problems, and adaptive skills—to validate the EIS. Missouri Performance Assessment state achievement scores, office disciplinary referrals, suspension and expulsion rates, were collected from school archival records each year of the project.

Data Analytic Strategy: For Study (1), the research team used confirmatory factor analysis and classification accuracy statistics. For Study (2), researchers coded focus group data by themes, and examined descriptive data for items on a social validity measure. For Study (3), the researchers conducted linear growth modeling and growth mixture modeling over a three period to determine changes in symptoms over time, identify developmental trajectories, and used latent class regression analysis to determine relationships between fidelity and student outcomes.

Related IES Projects: Enhancing the Capacity of Rural Schools to Identify, Prevent, and Intervene in Youth Mental Health Concerns: The National Center for Rural School Mental Health (R305C190014). Missouri Interdisciplinary Postdoctoral Research and Training Program (R305B150028).

Project website: http://bcschoolsmh.org

Products and Publications

Herman, K. C., Reinke, W. M., Huang, F. L., Thompson, A. M., & Doyle-Barker, L. (2020). An investigation of the psychometric properties of the early identification system-student report in a middle school sample. School Psychology.

Reinke, W.M., Herman, K. C., Thompson, A., McCall, C., Copeland, C., Holmes, S., & Owens, S. (2020). Investigating the longitudinal association between fidelity to a large-scale comprehensive school mental health prevention and intervention model and student outcomes. School Psychology Review.

Thompson, A. M., Huang, F., Smith, T., Reinke, W. M., & Herman, K. C. (2020). Confirmatory Factor Structure and Predictive Validity of the Early Identification System—Student Report in a Community Sample of High School Students. School Mental Health, 1–13.