Homelessness, teenage pregnancy, child abuse, parental substance abuse, and unsafe living conditions can put students at risk for negative academic outcomes.ⁱ Although these family- and community-related situations exist outside of schools, they can drastically affect students’ attendance, behavior, and course performance—the ABCs of early warning systems that schools often use to identify students at risk of dropping out.

According to the U.S. Department of Education, more than half of the high schools in the United States used early warning systems in 2014–2015, but most of these systems did not incorporate information on out-of-school risk factors such as homelessness, involvement in the criminal justice system, or involvement with child welfare services.ⁱⁱ

To determine whether adding information on these factors to early warning systems improves prediction of near-term academic risks, Regional Educational Laboratory (REL) Mid-Atlantic partnered with Pittsburgh Public Schools, the Propel Schools charter network, and Allegheny County Department of Human Services. Our goal was to assess how well combinations of in- and out-of-school longitudinal data predict the ABC risks that in turn predict dropout. The data included specific dates, months, or terms in which in- and out-of-school risk factors occurred. Importantly, we focused on predicting the ABCs that occur in the short term but did not aim to predict eventual dropout.

We found that predictive models have moderate to high accuracy using data that schools already have—demographics, eligibility for school services, and previous academic and behavior records. For example, for students in Pittsburgh Public Schools, there is an 80 percent chance that students the model flags are more at risk for chronic absenteeism than students the model does not flag. Adding information on out-of-school events from the human services agency (such as stays at a shelter or interactions with a child welfare agency) did not substantially change how successfully the system identified students at risk for near-term ABC problems. In other words, whether the system included out-of-school events or not did not make a significant difference!

This result does not mean that out-of-school events have no influence on academic issues. Rather, the out-of-school events do not provide new information beyond what school data, like receipt of free and reduced-price lunch or prior data on ABC risks, already provide with respect to improving predictions of which students are at risk for poor performance in school.

This finding points to some important implications for teachers and school administrators to consider:

• Schools can use data they likely already have to develop or maintain early warning systems to flag students at risk for the ABC problems with moderate to high accuracy. Across outcomes in Pittsburgh Public Schools, there is an 80 to 92 percent chance that students the predictive models flag are more at risk than students the models do not flag.
• Obtaining and maintaining data on out-of-school events can be resource intensive, and doing so will not improve the accuracy of predicting which students will experience the ABC problems.
• Even though out-of-school data from human services agencies do not improve predictions as to which students are at risk, educators could use such data to better understand some of the underlying challenges students face in their home lives and how to best support students when they are identified as at risk—in other words, emphasize the “why” instead of the “who.”

More research would help us confirm our finding that analyzing out-of-school circumstances does not improve the accuracy of early warning systems. In particular, future research could use data from schools or districts in a different context than Allegheny County to see if the results hold. Additional research could also test alternative data elements or predictive models, including models to predict eventual school dropout rather than the shorter-term ABC problems that often precede dropout. At REL Mid-Atlantic, we are continuing to investigate ways to leverage data schools and districts already have to inform approaches to reducing dropout and supporting academic success.