Skip Navigation
Skip Navigation

Back to Ask A REL Archived Responses

REL Midwest Ask A REL Response

Early Warning Systems - Behavior

June 2017

Question:

What does the research say about evidence-based practices or interventions that improve student behavior?



Response:

Following an established Regional Educational Laboratory (REL) Midwest protocol, we conducted a search for research reports and descriptive studies on evidence-based practices or interventions that improve student behavior, an early warning system indicator. In particular, we focused on identifying resources related to suspension and expulsion at the secondary level. For details on the databases and sources, keywords and selection criteria used to create this response, please see the methods section at the end of this memo.

Below, we share a sampling of the publicly accessible resources on this topic. The search conducted is not comprehensive; other relevant references and resources may exist. We have not evaluated the quality of references and resources provided in this response, but offer this list to you for your information only.

Research References

Bass, G., Lee, J., Wells, C., Carey, J. C., & Lee, S. (2015). Development and factor analysis of the protective factors index: A report card section related to the work of school counselors. Professional Counselor, 5(4), 516–528. Retrieved from https://eric.ed.gov/?id=EJ1109756

From the ERIC abstract: "The scale development and exploratory and confirmatory factor analyses of the Protective Factor Index (PFI) is described. The PFI is a 13-item component of elementary students' report cards that replaces typical items associated with student behavior. The PFI is based on the Construct-Based Approach (CBA) to school counseling, which proposes that primary and secondary prevention activities of school counseling programs should focus on socio-emotional, development-related psychological constructs that are associated with students' academic achievement and well-being, that have been demonstrated to be malleable, and that are within the range of expertise of school counselors. Teachers use the PFI to rate students' skills in four construct-based domains that are predictive of school success. School counselors use teachers' ratings to monitor student development and plan data-driven interventions."

Bergh, B., & Cowell, J. (2013). Discipline referral outcomes: Meeting the needs of students. Education Leadership Review, 14(3), 12-21. Retrieved from https://eric.ed.gov/?id=EJ1105374

From the ERIC abstract: "In this study, school disciplinary procedures/programs used in response to disciplinary referrals and programs that schools have in place to prevent, intervene, and respond to behaviors that result in suspensions or expulsions from the classroom or school were investigated. The focus of this study was school leaders' concerns regarding the effectiveness of the programs and procedures currently in place within their schools. Identification of programs to address the specific needs of special education students that are used in addition to or separate from the programs used with the general education population was an additional focus. The use of reintegration programs to assist any suspended or expelled student in successfully returning to the classroom or school environment was examined. Students receiving special education services incur a disproportionate number of discipline infractions, and often school remedial behavior programs do not work effectively to change behavior (Losen & Gillespies, 2012; Rose, Monda-Amaya, & Espelage, 2010; Skiba, Peterson, & Williams, 1997)."

Brown, A., & Lee, J. (2014). School performance in elementary, middle, and high school: A comparison of children based on HIPPY participation during the preschool years. School Community Journal, 24(2), 83–106. Retrieved from https://eric.ed.gov/?id=EJ1048627

From the ERIC abstract: "The purpose of this study was to determine the impact of the Home Improvement for Preschool Youngsters (HIPPY) program on school performance during the 3rd, 5th, 7th, and 9th grades. The study employed a quasi-experimental, post-hoc design using existing data on children who participated in the HIPPY program as 3-, 4-, or 5-year-olds, including: Texas Assessment of Knowledge and Skills (TAKS) scores, attendance records, school retention, and discipline referrals. Independent samples t-tests and chi-square analysis revealed that in all four grades HIPPY children had significantly higher rates of school attendance, were retained less often, had fewer repeat discipline referrals, scored higher, and had higher pass rates on the Reading and Math TAKS than matching children without HIPPY experience. Results indicate that children who participated in the HIPPY program as a 3-, 4-, or 5-year-old appear to have benefited long-term from the experience. The results also suggest that the HIPPY program intervention can increase school achievement and build a strong base for school success."

Christofferson, R. D., & Callahan, K. (2015). Positive behavior support in schools (PBSIS): An administrative perspective on the implementation of a comprehensive school-wide intervention in an urban charter school. Education Leadership Review of Doctoral Research, 2(2), 35-49. Retrieved from https://eric.ed.gov/?id=EJ1105721

From the ERIC abstract: "This research explores the implementation of a school-wide intervention program that was designed to foster and instill intrinsic values based on an external reward system. The Positive Behavior Support in Schools (PBSIS) is an intervention intended to improve the climate of schools using system-wide positive behavioral interventions to discourage disruptive behaviors. The charter school that was the focus of this research experienced high staff turnover, negative school climate and student suspension rates that exceeded the state average. A mixed methods research design included de-identified data that were retrieved from 200 students in grades kindergarten through two, 205 parents and 54 staff members. The data sources included data from the School-wide Evaluation Tool (SET), Climate Survey and Office Discipline Referrals (ODR). Results indicated the implementation of Positive Behavior Support in Schools had a positive and significant impact on improving student behaviors and school climate. Results indicated that the implementation of the program significantly reduced the number of office discipline referrals and in-school suspension rates, and improved perceptions of students, staff and parents regarding the school climate. However, the results also indicated that there was no significant difference in the out-of-school suspension rates during the two-year implementation of PBSIS. This study provided administrators and staff with a comprehensive understanding of the implementation challenges associated with a school-wide intervention, as well as evidence to support practices that were effective."

Collins, J. C., & Ryan, J. B. (2016). Extension of positive behavioral interventions and supports from the school to the bus: A case study. Journal of At-Risk Issues, 19(1), 29–33. Retrieved from https://eric.ed.gov/?id=EJ1104428

From the ERIC abstract: "Positive Behavioral Interventions and Supports (PBIS) is an evidence-based practice that has been shown to prevent and remediate challenging student behaviors, while concurrently improving academic outcomes. While the implementation of PBIS is a schoolwide process which involves multiple intensive trainings for all instructional and support staff, the vast majority of studies to date have focused on problem behaviors occurring within the school house, in either structured (e.g., classroom) or unstructured (e.g., playground) settings. This study extended the provision of common PBIS strategies and training components to bus drivers, with the goal of reducing challenging student behaviors during times of transit to and from school. Results revealed a substantial reduction of bus discipline referrals at the middle school level, while receiving high levels of satisfaction from both the bus drivers and school administrators. Additional findings and suggestions for future implementation are provided."

Faria, A. M., Sorensen, N., Heppen, J., Bowdon, J., Taylor, S., Eisner, R., & Foster, S. (2017). Getting students on track for graduation: Impacts of the Early Warning Intervention and Monitoring System after one year (REL 2017–272). Washington, DC: U.S. Department of Education, Institute of Education Sciences, National Center for Education Evaluation and Regional Assistance, Regional Educational Laboratory Midwest. Retrieved from https://eric.ed.gov/?id=ED573814

From the ERIC abstract: "Although high school graduation rates are rising—the national rate was 82 percent during the 2013/14 school year (U.S. Department of Education, 2015)-dropping out remains a persistent problem in the Midwest and nationally. Many schools now use early warning systems to identify students who are at risk of not graduating, with the goal of intervening early to help students get back on track for on-time graduation. Although research has guided decisions about the types of data and indicators used to flag students as being at risk, little is known about the impact of early warning systems on students and schools-and in particular, whether these systems do help get students back on track. This study, designed in collaboration with the REL Midwest Dropout Prevention Research Alliance, examined the impact and implementation of one early warning system-the Early Warning Intervention and Monitoring System (EWIMS)—on student and school outcomes... The study found that EWIMS reduced the percentage of students with risk indicators related to chronic absence and course failure but not related to low GPAs or suspension."

Hickman, G. P., & Wright, D. (2011). Academic and school behavioral variables as predictors of high school graduation among at-risk adolescents enrolled in a youth-based mentoring program. Journal of At-Risk Issues, 16(1), 25-33. Retrieved from https://eric.ed.gov/?id=EJ942899

From the ERIC abstract: "Using official school data, this study examined a sample of 447 at-risk students enrolled over a 10-year period in a youth-based mentoring program. The primary objective of the program was to ensure high school graduation. Participants were identified by indices of academic and school behaviors that rendered them less likely to graduate from high school. This study used logistic regression to examine the extent to which academic (i.e., GPA, grade retention, and math and reading proficiency scores) and behavioral (i.e., expulsions) variables, as well as age at entry of program, and duration in the program predicted high school graduation. Results indicated that GPA and participants' age at time of enrollment in the program were significant predictors of graduating high school. Implications are drawn for designers of diversion, intervention, and mentoring programs."

Johnson, K. C., & Lampley, J. H. (2010). Mentoring at-risk middle school students. SRATE Journal, 19(2), 64–69. Retrieved from https://eric.ed.gov/?id=EJ948699

From the ERIC abstract: "This study examined a mentoring program entitled: LISTEN (Linking Individual Students To Educational Needs). The LISTEN mentoring program was a district-sponsored, school-based program in which at-risk, middle school students were identified by the school system and mentors were recruited specifically to assist these students with school performance or related issues. Archival data from the 2003-04 and 2004-05 academic years were collected to determine the possible effects of the LISTEN mentoring program on at-risk students in grades six through eight. Specifically, the study investigated the relationship of a mentoring program with at-risk students' GPAs, discipline referrals, and attendance records. A statistically significant difference was found for GPAs, discipline referrals, and attendance rates between those measured pre-intervention and those measured post-intervention."

McCormick, M. P., Cappella, E., O'Connor, E. E., & McClowry, S. G. (2015). Do intervention impacts on academic achievement vary by school climate? Evidence from a randomized trial in urban elementary schools. Paper presented at the Society for Research on Educational Effectiveness Conference, Washington, DC. Retrieved from https://eric.ed.gov/?id=ED562123

From the ERIC abstract: "Given established links between social-emotional skills and academic achievement, there is growing support for implementing universal social/behavioral interventions in early schooling (Jones & Bouffard, 2012). Advocates have been particularly interested in implementing such programming in low income urban schools where students are likely to start school with lower levels of social-emotional and academic skills than their more affluent peers (Jones & Bouffard, 2012; Raver, 2002). There is inconsistent evidence, however, that such programs improve students' academic achievement over and above typical educational practice (SRCDC, 2010)... The current study is one of the first to consider the role of school climate in understanding moderated impacts of social/behavioral interventions on student achievement, attention, and behaviors. The major lesson from this work is that context matters. Across student outcomes, program impacts on achievement were generally larger, and sometimes driven by, schools that had less leadership, accountability and safety/respect prior to implementation of the intervention. Perhaps the biggest lesson from this study is for policymakers, who are currently engaged in distributing funding to expand and implement social/behavioral interventions in a variety of settings across the country."

Schmidt, A., & Canela, C. (2015). The behavioral outcomes of a self-affirmation intervention for middle school students. Paper presented at the Society for Research on Educational Effectiveness Conference, Washington, DC. Retrieved from https://eric.ed.gov/?id=ED562496

From the ERIC abstract: "Social psychological interventions in schools have gained popularity in education research for their ability to often dramatically increase student academic performance through simple exercises. Many of these interventions are designed to address stereotype threat, which is defined as 'being at risk of confirming, as self-characteristic, a negative stereotype about one's group' (Steele & Aronson, 1995, p. 797). One of the social psychological interventions with the most drastic impact on the performance of students potentially affected by stereotype threat (i.e., African American students) was a set of self-affirmation exercises administered in racially diverse middle schools by Cohen and his colleagues (Cohen, Garcia, Apfel, & Master, 2006; Cohen, Garcia, Purdie-Vaughns, Apfel, & Brzustoski, 2009). The intervention, a brief writing exercise about why a value the student selected (e.g., friendship) is important to him or her, resulted in a 40 percent decrease in the racial grade point average gap in the year of administration (Cohen, Garcia, Apfel, & Master, 2006), as well as long-term increases in the grade point averages of African American students (Cohen, Garcia, Purdie-Vaughns, Apfel, & Brzustoski, 2009). While Cohen and his colleagues' (2006; 2009) self-affirmation intervention focused on academic achievement outcomes, there is evidence suggesting the intervention may also be able to elicit positive behavioral responses in students by supplementing their levels of self-control. Based on these findings, the authors sought to determine if a self-affirmation intervention can influence the behavior of middle school students over the course of three years, focusing on a measure of behavior that is both common in middle/high school settings and potentially related to students' ability to self-regulate: the number of office discipline referrals (ODRs) students receive over the course of a school year. For now, the authors found no strong evidence to suggest that self-affirmation exercises should be adopted as a strategy to address office discipline referral receipt in schools."

U.S. Department of Education, Institute of Education Sciences, What Works Clearinghouse. (2016, December). Children Identified With or At Risk for an Emotional Disturbance topic area intervention report: Functional behavioral assessment-based interventions. Retrieved from https://eric.ed.gov/?id=ED572042

From the ERIC abstract: "This intervention report presents findings from a systematic review of 'functional behavioral assessment-based interventions' conducted using the WWC Procedures and Standards Handbook, version 3.0, and the Children Identified With or At Risk for an Emotional Disturbance review protocol, version 3.0. Functional behavioral assessment (FBA) is an individualized problem-solving process for addressing student problem behavior. An assessment is conducted to identify the purpose or function of a student's problem behavior. This assessment process involves collecting information about the environmental conditions that precede the problem behavior and the subsequent rewards that reinforce the behavior. The information that is gathered is then used to identify and implement individualized interventions aimed at reducing problem behaviors and increasing positive behaviors. The What Works Clearinghouse (WWC) identified 17 studies of FBA-based interventions that both fall within the scope of the Children Identified with or At Risk for an Emotional Disturbance topic area and meet WWC pilot single-case design standards. Seven studies meet pilot single-case design standards without reservations, and 10 studies meet pilot single-case design standards with reservations. Together, these single-case design studies included 39 children between 5 and 18 years old who are identified with or at risk for an emotional disturbance. Those 17 studies are summarized in this report."

U.S. Department of Education, Institute of Education Sciences, What Works Clearinghouse. (2015, May). Dropout Prevention intervention report: Check & Connect. Retrieved from https://eric.ed.gov/?id=ED556120

From the ERIC abstract: "'Check & Connect' aims to help students stay in school by continually monitoring school performance and providing individualized attention through mentoring, case management, and other supports. In 2006, the WWC published a systematic review of all the studies that examined the impact of 'Check & Connect' on high school students with learning, behavioral, or emotional disabilities who are at risk of dropping out. The WWC recently updated this report to include more recent publications. Based on the most up-to-date evidence, the WWC found that 'Check & Connect' has positive effects on staying in school, potentially positive effects on progressing in school, and no discernible effects on completing school for high school students with learning, behavioral, or emotional disabilities."

U.S. Department of Education, Institute of Education Sciences, What Works Clearinghouse (2011, October). Intervention report: Coping power. Retrieved from https://eric.ed.gov/?id=ED525365

From the ERIC abstract: "'Coping Power' is based on the earlier 'Anger Coping Power' program. It emphasizes social and emotional skills that are needed during the transition to middle school. The program incorporates child and parent components. The child component consists of thirty-four 50-minute group sessions and periodic individual sessions over the course of 15-18 months, although the program can be shortened to fit into a single school year. Lessons focus on goal setting, problem solving, anger management, and peer relationships. The parent component is composed of 16 group sessions and periodic individual meetings. Lessons support the child component of the program and address setting expectations, praise, discipline, managing stress, communication, and child study skills. Three studies of 'Coping Power' that fall within the scope of the Children Classified as Having an Emotional Disturbance review protocol meet What Works Clearinghouse (WWC) evidence standards. The three studies included 650 students who were at high risk for delinquent and/or aggressive behavior from grades 4 and 5 in Alabama and North Carolina. Based on these three studies, the WWC considers the extent of evidence for 'Coping Power' on children classified as having an emotional disturbance (or children at risk for classification) to be medium to large for external behavior and small for social outcomes. The three studies that meet WWC evidence standards did not examine the effectiveness of 'Coping Power' on children classified with an emotional disturbance in the emotional/internal behavior, reading achievement/literacy, math achievement, school attendance, or other academic performance domains. 'Coping Power' was found to have positive effects on external behavior and potentially positive effects on social outcomes for children classified with an emotional disturbance. Three studies (Lochman et al., 2009; Lochman, Boxmeyer, Powell, Roth, & Windle, 2006; Lochman & Wells, 2004) are randomized controlled trials that meet WWC evidence standards. The remaining 20 studies do not meet WWC eligibility screens or WWC evidence standards."

Additional Organizations to Consult

Early Warning Systems Pathway – http://www.earlywarningsystems.org

From the website: "AIR focuses on ensuring that an early warning system (EWS) is used to take action. We guide leaders and educators through a research-based process of identifying at-risk students, targeting supports, and monitoring students to improve graduation outcomes. Our knowledge of EWS, including the development of tools and resources to support implementation, and our extensive experience in providing EWS design, technical assistance, and training to stakeholders makes AIR an ideal partner in supporting at-risk students."

What Works Clearinghouse – https://ies.ed.gov/ncee/wwc/FWW/Results?filters=,Behavior

From the website: "The What Works Clearinghouse (WWC) reviews the existing research on different programs, products, practices, and policies in education. Our goal is to provide educators with the information they need to make evidence-based decisions. We focus on the results from high-quality research to answer the question ‘What works in education?'"

Methods

Keywords and Search Strings

The following keywords and search strings were used to search the reference databases and other sources:

  • practices OR interventions AND behavior-student behavior

  • practices OR interventions AND suspension

  • practices OR interventions AND discipline referrals

Databases and Search Engines

We searched ERIC for relevant resources. ERIC is a free online library of more than 1.6 million citations of education research sponsored by the Institute of Education Sciences (IES).

Reference Search and Selection Criteria

When we were searching and reviewing resources, we considered the following criteria:

  • Date of the publication: References and resources published over the last 15 years, from 2002 to present, were include in the search and review.

  • Search priorities of reference sources: Search priority is given to study reports, briefs, and other documents that are published or reviewed by IES and other federal or federally funded organizations.

  • Methodology: We used the following methodological priorities/considerations in the review and selection of the references: (a) study types—randomized control trials, quasi-experiments, surveys, descriptive data analyses, literature reviews, policy briefs, and so forth, generally in this order, (b) target population, samples (e.g., representativeness of the target population, sample size, volunteered or randomly selected), study duration, and so forth, and (c) limitations, generalizability of the findings and conclusions, and so forth.
This memorandum is one in a series of quick-turnaround responses to specific questions posed by educational stakeholders in the Midwest Region (Illinois, Indiana, Iowa, Michigan, Minnesota, Ohio, Wisconsin), which is served by the Regional Educational Laboratory (REL Region) at American Institutes for Research. This memorandum was prepared by REL Midwest under a contract with the U.S. Department of Education’s Institute of Education Sciences (IES), Contract ED-IES-17-C-0007, administered by American Institutes for Research. Its content does not necessarily reflect the views or policies of IES or the U.S. Department of Education nor does mention of trade names, commercial products, or organizations imply endorsement by the U.S. Government.