Co-Principal Investigator: Robert Horner

Purpose: Although many effective interventions are adopted in today's schools, they rarely sustain beyond a year or two once external support (e.g., grant funding, university-based training) is removed. The existing literature on sustainability of practices in schools is primarily anecdotal and not based on empirical research. Accordingly, school personnel are left without guidance on how to implement sustainable practices. Because so little attention has been devoted to the rigorous study of implementation and sustainability, new empirical research is needed to understand the best ways to promote implementation and sustainability of effective interventions.

The purpose of this study is to identify malleable factors that enhance or inhibit the implementation and sustainability of school-wide social-emotional and behavior support practices. The research will focus on School-wide Positive Behavioral Interventions and Supports (PBIS) because it has been widely adopted in the United States, has sustained in many schools, and relies on validated fidelity of implementation measures that can be used to document sustained implementation.

Project Activities: In partnership with five existing state PBIS implementation networks, the research team proposes a line of research using primary and secondary data analysis to predict implementation and sustainability of school-wide behavior support practices. The project will occur in three phases. Phase I involves validating a model of sustainable implementation of school-wide interventions as it applies to PBIS by testing the factor structure of a measure of contextual variables related to implementation and sustainability. These data will then be used in a longitudinal analysis in Phase III. Phase II includes secondary data analysis of PBIS fidelity of implementation data. Analyses will identify critical school demographic variables and school team actions that predict initial and sustained implementation. Phase III uses a longitudinal design to examine predictors of implementation and sustainability for schools at three key points of implementation (initial implementation, 3 years of implementation, and 5 years of implementation).

Products: The expected products from this study include publications and presentations on research activities and findings that may serve as a basis for developing interventions for schools to implement and sustain effective interventions.

Structured Abstract

Setting: Data will be collected from five states (Maryland, Minnesota, Missouri, Washington, Wisconsin) with established PBIS initiatives and ongoing training for schools adopting and sustaining PBIS. School demographic data are publically available for download from the Common Core of Data provided by the National Center for Education Statistics in the Institute of Education Sciences. Data regarding year of PBIS initial implementation and fidelity of implementation are available to the project team in the PBIS research database at the University of Oregon. Schools implementing PBIS who have agreed to share their data for research purposes become part of an extensive database housed at the University of Oregon.

Sample: For the secondary data analysis of the PBIS database, a sample of at least 514 schools that began implementing PBIS and received evaluations of fidelity of implementation at 3 years of implementation and/or at 5 years of implementation will be included in the analyses.

For the longitudinal sample, school team leaders and district coaches from 750 elementary and secondary public schools implementing PBIS will participate. Three separate samples of schools will be recruited (50 per state per sample), with samples of schools representing each of three critical stages of implementation: initial implementation, full operation (3 years), and sustainability (5 years). Schools must also be using the School-wide Information System, a web-based computer application that PBIS schools in the partner states use to track student discipline.

Intervention: The research team is examining PBIS, a systems-level approach used in over 14,500 schools to improve school climate and safety by teaching students prosocial behavior and redesigning school environments to discourage problem behavior.

Research Design and Methods: The plan includes a series of studies, conducted over three project phases, to achieve the following research aims: (a) validate the model of sustainable implementation of school-wide interventions as it applies to PBIS by testing the factor structure of a measure of contextual variables related to implementation and sustainability (Phase I); (b) conduct secondary data analysis and use a longitudinal design to identify malleable contextual and practice variables that predict initial and sustained implementation (Phases II and III); and (c) refine the proposed theory of change intended to assist in the development of a promising systems-level intervention to enhance implementation and sustainability of PBIS and other school-wide practices (Phase III).

Control Condition: Due to the nature of the research design, there is no control condition.

Key Measures: School demographic data accessed from extant NCES databases include enrollment, grade levels served, student-teacher ratio, and school type (e.g., regular, alternative, juvenile justice, charter, magnet). The School-wide Universal Behavior Sustainability Index: School Teams (SUBSIST) will measure contextual and practice variables predicting implementation and sustainability of school-wide behavior support practices (e.g., programs, curricula) delivered to all students. The School-wide Evaluation Tool (SET) will measure fidelity of implementation of PBIS. Additional school self-report measures of PBIS implementation will also be examined.

Data Analytic Strategy: Data analysis will include multi-level latent growth curve modeling and mediation analyses using bootstrapping. In all analyses, the team will account for nested data structures, use contemporary missing data handling, and compare model fit with alternative models.

Products and Publications

Book chapter

McIntosh, K., and Turri, M.G. (2014). Positive Behavior Support: Sustainability and Continuous Regeneration. In C.R. Reynolds, K.J. Vannest, and E. Fletcher-Janzen (Eds.), Encyclopedia of Special Education: A Reference for the Education of Children, Adolescents, and Adults With Disabilities and Other Exceptional Individuals (pp. 2061–2064). Hoboken, NJ: Wiley.

McIntosh, K., Lucyshyn, J.M., Strickland-Cohen, M.K., and Horner, R.H. (2014). Building Supportive Environments: Toward a Technology for Enhancing Fidelity of Implementation. In F. Brown, J. Anderson, and R.L. DePry (Eds.), Individual Positive Behavior Supports: A Standards-based Guide to Practices in School and Community-based Settings (pp. 413–428). Baltimore, MD: Brookes.

Nese, R.N.T., and McIntosh, K. (2016). Do School-Wide Positive Behavioral Interventions and Supports, Not Exclusionary Discipline Practices. In B.G. Cook, Tankersley, M., and T.J. Landrum (Eds.), Advances in Learning and Behavioral Disabilities, Volume 29 (pp. 175–196). Bingley, UK: Emerald.

Journal article, monograph, or newsletter

Andreou, T. E., McIntosh, K., Ross, S. W., & Kahn, J. D. (2015). Critical incidents in sustaining school-wide positive behavioral interventions and supports. The Journal of Special Education, 49(3), 157–167.

Mathews, S., McIntosh, K., Frank, J.L., and May, S. (2014). Critical Features Predicting Sustained Implementation of School-Wide Positive Behavioral Interventions and Supports. Journal of Positive Behavior Interventions, 16(3): 168–178. doi:10.1177/1098300713484065 Full text

McIntosh, K., Kelm, J.L., and Canizal Delabra, A. (2016). In Search of How Principals Change: A Qualitative Study of Events That Help and Hinder Administrator Support for School-Wide PBIS. Journal of Positive Behavior Interventions, 18(2): 100–110. doi:10.1177/1098300715599960 Full text

McIntosh, K., Kim, J., Mercer, S.H., Strickland-Cohen, M.K., and Horner, R.H. (2015). Variables Associated With Enhanced Sustainability of School-Wide Positive Behavioral Interventions and Supports. Assessment for Effective Intervention, 40(3): 184–191. doi:10.1177/1534508414556503 Full text

McIntosh, K., Mercer, S.H., Nese, R.N.T., and Ghemraoui, A. (in press). Patterns of Implementation of a Scaled-Up School-Based Prevention Model Over a Five-Year Period. Prevention Science.

McIntosh, K., Mercer, S.H., Nese, R.N.T., Strickland-Cohen, M.K., and Hoselton, R. (2016). Predictors of Sustained Implementation of School-Wide Positive Behavioral Interventions and Supports. Journal of Positive Behavior Interventions, 18(4): 209–218. doi:10.1177/1098300715599737

McIntosh, K., Predy, L.K., Upreti, G., Hume, A.E., Turri, M.G., and Mathews, S. (2014). Perceptions of Contextual Features Related to Implementation and Sustainability of School-Wide Positive Behavior Support. Journal of Positive Behavior Interventions, 16(1): 31–43. doi:10.1177/1098300712470723

McIntosh, K., Ty, S.V., and Miller, L.D. (2014). Effects of School-Wide Positive Behavioral Interventions and Supports on Internalizing Problems: Current Evidence and Future Directions. Journal of Positive Behavior Interventions, 16(4): 209–218. doi:10.1177/1098300713491980 Full text

Mercer, S.H., McIntosh, K., Strickland-Cohen, M.K., and Horner, R.H. (2014). Measurement Invariance of an Instrument Assessing Sustainability of School-Based Universal Behavior Practices. School Psychology Quarterly, 29(2): 125–137. doi:10.1037/spq0000054 Full text

Nese, R.N.T., McIntosh, K., Nese, J.F.T., Bloom, J., Johnson, N.W., Phillips, D., and Hoselton, R. (in press). Predicting Abandonment of School-Wide Behavior Interventions. Behavioral Disorders.

Pinkelman, S., McIntosh, K., Rasplica, C., Berg, T., and Strickland-Cohen, M.K. (2015). Perceived Enablers and Barriers Related to Sustainability of School-Wide Positive Behavioral Interventions and Supports. Behavioral Disorders, 40(3): 171–183. Full text

Schaper, A., McIntosh, K., and Hoselton, R. (2016). Within-Year Fidelity Growth of SWPBIS During Installation and Initial Implementation. School Psychology Quarterly, 31(3): 358–368. doi:10.1037/spq0000125

Strickland-Cohen, M.K., McIntosh, K., and Horner, R.H. (2014). Sustaining Effective Practices in the Face of Principal Turnover. Teaching Exceptional Children, 46(3): 18–24. Full text

Turri, M.G., Mercer, S.H., McIntosh, K., Nese, R.N.T., Strickland-Cohen, M.K., and Hoselton, R. (in press). Examining Barriers to Sustained Implementation of School-Wide Prevention Practices. Assessment for Effective Intervention. doi:10.1177/1534508416634624