Skip Navigation
archived information

Implementation of Early Warning Systems
June 2021

Question

"What does the research say about the implementation of early warning systems for the high school level?"

Ask A REL Response

Thank you for your request to our Regional Educational Laboratory (REL) Reference Desk. Ask A REL is a collaborative reference desk service provided by the 10 RELs that, by design, functions much in the same way as a technical reference library. Ask A REL provides references, referrals, and brief responses in the form of citations in response to questions about available education research.

Following an established REL Northwest research protocol, we conducted a search for evidence- based research. The sources included ERIC and other federally funded databases and organizations, research institutions, academic research databases, Google Scholar, and general Internet search engines. For more details, please see the methods section at the end of this document.

The research team has not evaluated the quality of the references and resources provided in this response; we offer them only for your reference. The search included the most commonly used research databases and search engines to produce the references presented here. References are listed in alphabetical order, not necessarily in order of relevance. The research references are not necessarily comprehensive and other relevant research references may exist. In addition to evidence-based, peer-reviewed research references, we have also included other resources that you may find useful. We provide only publicly available resources, unless there is a lack of such resources or an article is considered seminal in the topic area.

References

Allensworth, E. (2013). The use of ninth-grade early warning indicators to improve Chicago schools. Journal of Education for Students Placed at Risk (JESPAR), 18(1), 68-83. Retrieved from https://www.shankerinstitute.org

From the Abstract:
"Chicago has been in the forefront of the country in its use of 9th-grade indicators of dropout. Catalyzed by the development of the freshman on-track indicator and research around it, Chicago school administrators, central office personnel, and external partners have developed a number of mechanisms using 9th-grade indicators to stimulate school improvement. This article describes 3 ways in which early warning indicators are useful for improving student achievement: (a) focusing conversations and efforts on actionable problems; (b) identifying students for intervention; and (c) using indicator patterns to address low performance in a strategic way. Examples from high schools in Chicago suggest that knowledge of the on-track indicator and its use in district accountability were not enough to change practice. However, the availability of data tools that make it easy to act on information about on-track rates have changed the ways in which teachers and school staff interact with each other, students, and parents regarding improving student performance. The strategies they have developed with the data tools have provided a systematic focus to their efforts, which appears to be paying off in substantially improved ninth-grade achievement."

Bruce, M., Bridgeland, J. M., Fox, J. H., & Balfanz, R. (2011). On Track for Success: The use of early warning indicator and intervention systems to build a grad nation. Civic Enterprises. https://eric.ed.gov

From the Abstract:
"Over the past decade, schools, districts, and states have become increasingly savvy with data collection and analysis to drive student outcomes. The development and use of Early Warning Indicator and Intervention Systems (EWS) are at the cutting edge of the data- driven, outcomes-focused, high-impact education movement. These systems can increase educators' effectiveness by helping them use data to identify those students who are on track to graduate, and those who are falling behind, far enough in advance to provide appropriate interventions. This report represents the first national assessment of EWS at the district, state, and national levels. It shares evidence from the latest research and best practices from the field so that parents, educators, administrators, business leaders, and legislators can be better equipped to keep children on track to graduate high school, prepared for college and career success."

Davis, M., Herzog, L., & Legters, N. (2013). Organizing schools to address early warning indicators (EWIs): Common practices and challenges. Journal of Education for Students Placed at Risk (JESPAR), 18(1), 84-100. Retrieved from http://www.pathwaystoadultsuccess.org

From the Abstract:
"An early warning system is an intentional process whereby school personnel collectively analyze student data to monitor students at risk of falling off track for graduation and to provide the interventions and resources to intervene. We studied the process of monitoring the early warning indicators and implementing interventions to ascertain common practices and challenges. Research questions focused on the process of collecting and reviewing student data, conducting team meetings, and designing or selecting, and following through on, student interventions. This article reports on typical practices and implementation challenges of early warning indicators meetings from 10 schools that participated in the pilot phase of the Diplomas Now School Transformation Model across the country, gathered through meeting observations and interviews."

Davis, M. H., Mac Iver, M. A., Balfanz, R. W., Stein, M. L., & Fox, J. H. (2019). Implementation of an early warning indicator and intervention system. Preventing School Failure: Alternative Education for Children and Youth, 63(1), 77-88. https://eric.ed.gov

From the Abstract:
"This study focuses on the implementation of an early warning indicator and intervention system in 20 southern high schools. This model included a team of teachers, counselors, and student support services personnel who analyzed ninth grade student-level data and implemented and monitored interventions. The team was led by a half-time coach who established protocols for team operations and provided updated early warning indicator data. Findings indicated variability in the frequency and scale of the interventions initiated and monitored by the promotion coaches and early warning indicator teams. This study further demonstrates that higher levels of implementation of early warning and intervention components in the treatment schools were associated with improved levels of ninth grade student attendance and course performance even after controlling for prior attendance." [This paper was published in "Preventing School Failure" v63 n1 p77-88 2019 (EJ1209816).

Faria, A.-M., Sorensen, N., Heppen, J., Bowdon, J., Taylor, S., Eisner, R., & Foster, S. (2017). Getting students on track for graduation: Impacts of the Early Warning Intervention and Monitoring System after one year (REL 2017–272). U.S. Department of Education, Institute of Education Sciences, National Center for Education Evaluation and Regional Assistance, Regional Educational Laboratory Midwest. https://eric.ed.gov

From the Abstract:
"This study, designed in collaboration with the REL Midwest Dropout Prevention Research Alliance, examined the impact and implementation of one early warning system--the Early Warning Intervention and Monitoring System (EWIMS)--on student and school outcomes. The study examined the impacts of EWIMS on indicators of student risk and on student progress in school after the first year of EWIMS adoption. This report provides rigorous initial evidence that even with limited implementation during the first year of adoption, using a comprehensive early warning system can reduce the percentage of students who are chronically absent or who fail one or more courses. These short-term results are promising because chronic absence and course failure in grades 9 and 10 are two key indicators that students are off track for on-time graduation. However, because the past research linking indicators to on-time graduation is correlational, it is not yet known if improving these indicators leads to improving on-time graduation rates. Also, EWIMS did not have a detectable impact on other measured indicators that are related to students' likelihood of on-time graduation, including low GPAs, suspensions, and earning insufficient credits. Future research is needed to better understand the mechanisms through which EWIMS had an impact on chronic absence and course failure and why EWIMS did not affect other outcomes."

Frazelle, S., & Nagel, A. (2015). A practitioner’s guide to implementing early warning systems. (REL 2015–056). U.S. Department of Education, Institute of Education Sciences, National Center for Education Evaluation and Regional Assistance, Regional Educational Laboratory Northwest. https://eric.ed.gov

From the Abstract:
"To stem the tide of students dropping out, many schools and districts are turning to early warning systems (EWS) that signal whether a student is at risk of not graduating from high school. While some research exists about establishing these systems, there is little information about the actual implementation strategies that are being used across the country. This report summarizes the experiences and recommendations of EWS users throughout the United States. A list of references and resources is provided."

Hartman, J., Wilkins, C., Gregory, L., Gould, L. F., & D’Souza, S. (2011). Applying an on-track indicator for high school graduation: Adapting the Consortium on Chicago School Research indicator for five Texas districts (REL 2011–100). U.S. Department of Education, Institute of Education Sciences, National Center for Education Evaluation and Regional Assistance, Regional Educational Laboratory Southwest. http://eric.ed.gov

From the Abstract:
"This study uses a measure of the on-track or off-track status of students at the end of grade 9 as an indicator of whether students in five Texas districts would graduate from high school in four years. In all five districts, on-time graduation rates were higher for students who were on track at the end of grade 9 than for students who were off track, both for students overall and for all racial/ethnic groups. This study is a first step in helping local districts and the Texas Education Agency develop an on-track indicator that accurately differentiates at the end of grade 9 between students who do and those who do not graduate on time. Appended are: (1) Study methodology; (2) District profiles; and (3) Off-track analysis."

Li, Y., Scala, J., Gerdeman, D., & Blumenthal, D. (2016). District guide for creating indicators for early warning system. U.S. Department of Education, Institute of Education Sciences, National Center for Education Evaluation and Regional Assistance, Regional Educational Laboratory West. Retrieved from https://ies.ed.gov

From the Abstract:
"Across the country, many K-12 school districts have implemented early warning systems that use readily available data to identify students who are at risk of failing to achieve a desired outcome, such as graduating from high school on time. These systems allow educators to intervene early to keep students on track. The guide lays out a seven-step process for creating local indicators, starting with reviewing possible measures, moving through exploratory analyses of local data, and examining the reliability of different indicators or composites to inform local decisions. At each step, templates are provided to illustrate how data can be analyzed and displayed. The templates are also available as Word documents so they can be tailored to local contexts. Developed by researchers at the American Institutes for Research, the guide draws on several years of work partnering with districts to help them look at their local data in new ways. Now others can follow a similar process."

McMahon, B. M., & Sembiante, S. F. (2020). Re‐envisioning the purpose of early warning systems: Shifting the mindset from student identification to meaningful prediction and intervention. Review of Education, 8(1), 266-301. Retrieved from https://www.researchgate.net

From the Abstract:
"Emphasis in school dropout literature has shifted from exploring wide-ranging causes of dropping out to soliciting a smaller number of predictive indicators to identify students at increased risk for dropping out. However, much of the past decade's Early Warning research excludes indicators that do not add to the predictive nature of the model even if they might inform intervention. Considering this shift and the relative infancy of Early Warning research, this paper presents a critical systematic review of the literature surrounding dropout and Early Warning System(s) (EWS). Through a focus and evaluation of student-level indicators, we present an argument of how an effective EWS can bridge the gap between the prediction of dropout events and underlying causes, while providing actionable information for schools to intervene. Finally, we present preliminary data that demonstrate the potential of elementary-level student-level indicators as an avenue for shifting the focus of EWS from student identification to meaningful prediction and intervention. The results show how the student-level indicators explored in this paper can effectively identify students at these grade levels who are on- and off-track to graduation. We discuss the implications of evidence warranting the position that dropping out of school can be predicted as early as the elementary grade levels and its potential to drive future research in this area."

Methods

Keywords and Search Strings: The following keywords, subject headings, and search strings were used to search reference databases and other sources: ("early warning system*" OR "early warning indicator*") AND "high school" AND implementation

Databases and Resources: We searched ERIC for relevant resources. ERIC is a free online library of more than 1.6 million citations of education research sponsored by the Institute of Education Sciences (IES). Additionally, we searched Google Scholar and EBSCO databases (Academic Search Premier, Education Research Complete, and Professional Development Collection).

Reference Search and Selection Criteria

When we were searching and reviewing resources, we considered the following criteria:

Date of publications: This search and review included references and resources published in the last 10 years.

Search priorities of reference sources: Search priority was given to study reports, briefs, and other documents that are published and/or reviewed by IES and other federal or federally funded organizations, as well as academic databases, including ERIC, EBSCO databases, and Google Scholar.

Methodology: The following methodological priorities/considerations were given in the review and selection of the references:

  • Study types: randomized control trials, quasi experiments, surveys, descriptive data analyses, literature reviews, and policy briefs, generally in this order
  • Target population and samples: representativeness of the target population, sample size, and whether participants volunteered or were randomly selected
  • Study duration
  • Limitations and generalizability of the findings and conclusions

This memorandum is one in a series of quick-turnaround responses to specific questions posed by stakeholders in Alaska, Idaho, Montana, Oregon, and Washington, which is served by the Regional Educational Laboratory (REL) Northwest. It was prepared under Contract ED-IES-17-C-0009 by REL Northwest, administered by Education Northwest. The content does not necessarily reflect the views or policies of IES or the U.S. Department of Education, nor does mention of trade names, commercial products, or organizations imply endorsement by the U.S. Government.