Grant Closed

Examining Variation in the Impact of School-Wide Positive Behavioral Interventions and Supports (PBIS)

NCER
Program: Education Research Grants
Program topic(s): Social, Emotional, and Behavioral Context for Teaching and Learning
Award amount: $700,000
Principal investigator: Catherine Bradshaw
Awardee:
Johns Hopkins University
Year: 2009
Project type:
Exploration
Award number: R305A090307

Purpose

Approximately one out of five children displays disruptive behavior problems and this rate may be even higher among children in urban communities. Behavior problems are of great concern in elementary school settings because of their associations with low rates of attendance, reduced academic achievement, and greater risk for special education placements. Students who exhibit problem behaviors early on are also at greater risk for later school failure, substance abuse, and antisocial behavior in adolescence and early adulthood. Positive Behavioral Interventions and Supports (PBIS) is a universal school-wide system to prevent disruptive problem behavior in schools. School-wide PBIS (SWPBIS) is currently being implemented in over 7,500 schools across 44 states. It provides schools with a mechanism for creating a more positive behavioral environment via systemic changes in school systems and procedures for handling disruptive behaviors. SWPBIS has shown positive effects on school-wide suspension rates and office disciplinary referrals, but teachers report that about 20% of their students need more targeted services. In this study, the research team will link existing data from the only longitudinal randomized controlled trial (RCT) of SWPBIS with archival data from a state department of education to identify for whom, how, and under what conditions SWPBIS was most effective. This work will inform the future development of screening tools and targeted preventive interventions for children who do not respond adequately to the universal SWPBIS model.

Project Activities

Data from a previous SWPBIS-RCT conducted in 37 elementary schools in Maryland will be linked to archival data from the Maryland State Department of Education. The research team will examine characteristics of students and schools that predict behavior problems under the SWPBIS model. Using a variety of latent variable modeling techniques, the team will (1) identify malleable behavioral, social-emotional, and academic characteristics of children who do not respond adequately to the SWPBIS model; (2) examine the influence of malleable contextual factors such as fidelity of implementation or schools' organizational health on variation in responsiveness to SWPBIS; and (3) identify factors that mediate the effect of SWPBIS on student outcomes.

Structured Abstract

Setting

The research team will use data collected from a 5-year longitudinal randomized controlled trial (RCT) of SWPBIS in Maryland and link it to archival student data from the Maryland State Department of Education from the corresponding time period.

Sample

Three cohorts comprised of 12,104 children in 37 elementary schools who were in kindergarten, first or second grade at the start of the RCT, and the 3,563 staff (55% general education teachers and 45% student support staff) in those study schools.
Intervention
School-wide Positive Behavioral Interventions and Supports (SWPBIS) has seven core features which are described here. Within the school, a PBIS team is formed that includes 6–10 staff members and an administrator, all of whom provide building-level leadership regarding the implementation of SWPBIS. A behavioral support "coach" provides on-site consultation and technical assistance regarding the implementation of PBIS. The coach is typically a school psychologist or guidance counselor who has prior experience working with PBIS and conducting functional behavioral assessments. Expectations for positive student behavior are defined and known by staff and students. The school team establishes 3–5 positively stated school-wide expectations for student behavior (e.g., "Be respectful, responsible, and ready to learn"), which are posted in all school settings. Defined behavioral expectations are taught to all students. A school-wide system is developed to reward students who exhibit the expected positive behaviors. An agreed upon system is created to respond to behavioral violations. Staff and administrators agree on what constitutes classroom versus office-managed discipline problems, and students across all classrooms receive consistent consequences for disciplinary infractions. A formal system for data-based decision-making is developed to collect, analyze, and use disciplinary data.

Research design and methods

Existing data from the SWPBIS-RCT will be linked to archival student-level disciplinary and academic data from the Maryland State Department of Education. The RCT was a 5-year study funded by the Centers for Disease Control and Prevention-National Institute of Mental Health involving 37 elementary schools, 21 randomly assigned to implement SWPBIS and 16 randomly assigned to a control group. Data were collected from all students in the school building during the first and last year of the RCT, and annually on three cohorts of children who were in kindergarten, first and second grade when the RCT began. Using the linked data sets, the research team will examine multiple indicators of non-response to SWPBIS at the student level (educational or disciplinary problems and service use or need for services) and at the school level (implementation quality and organizational health).

Control condition

There is no control condition.

Key measures

Information about implementation quality, school context, and student demographics and behavior are available from the RCT. SWPBIS implementation quality was measured using the School-Wide Evaluation Tool. The school environment was assessed using the Organizational Health Inventory, the School Climate Survey, and the Student Interaction in Specific Setting observation instrument. Classroom teachers participating in the RCT completed the Teacher-Report of Student Aggressive/Disruptive Behavior, Concentration Problems, and Prosocial Behavior (TOCA-C), a checklist version of the Teacher Observation of Classroom Adaptation. In addition to measuring these three specific classroom behaviors, the TOCA-C also includes items that measure teacher report of office discipline referrals (ODRs) and students' need for counseling and special education services, and asks teachers to make a global rating of each student's academic performance. ODRs were also measured by each school using the Internet-based School-Wide Information System. Three types of student-level data will be obtained from the MSDE for this study. Academic performance will be assessed using reading and math scores from the Maryland School Assessment administered in third, fourth and fifth grade, report card grades, and information about annual grade promotion/retention. Discipline problems will be assessed using individual students' official records of suspensions. Special education service use will be assessed using official records of Individual Education Plans.

Data analytic strategy

The primary analytic tool is latent variable modeling, a type of structural equation modeling (SEM). A variety of longitudinal modeling procedures will be performed as appropriate for each research question, including latent class/profile analysis, SEM with dichotomous outcomes, general growth mixture modeling, receiver operator curves, survival analysis, and multilevel modeling.

People and institutions involved

IES program contact(s)

Emily Doolittle

Team Lead for Social Behavioral Research
NCER

Products and publications

Products: The expected products of this study include information about the malleable student and school characteristics that predict student non-response to SWPBIS. This information will guide the development of screening tools and more targeted interventions to help schools manage student behaviors in ways to support learning and academic success.

Book chapter

Bottiani, J., and Bradshaw, C.P. (2012). Social-Emotional Learning. In E. Anderman, and J. Hattie (Eds.), International Guide to Student Achievement (pp. 173-176). New York: Routledge.

Debnam, K.J., Bradshaw, C.P., Pas, E.T., and Lindstrom-Johnson, S. (2015). School as a Unit of Assignment and Analysis in Group-Randomized Controlled Trials. Prevention Science in School Settings: Complex Relationships and Processes (pp. 247-269). New York: Springer.

Mendelson, T., Pas, E.T., Bradshaw, C.P., Leis, J., Leaf, P.J., and Rebok, G. (2012). The Logic and Practice of Prevention. In W. Eaton (Ed.), Public Mental Health (pp. 459-509). New York: Oxford.

Pas, E.T., and Newman, D.L. (2013). Teacher Mentoring, Coaching, and Consultation. In J.A.C. Hattie, and E.M. Anderman (Eds.), International Guide to Student Achievement (pp. 152-154). New York: Routledge Publishing Company.

Sugai, G., Simonsen, B., Bradshaw, C., Horner, R., and Lewis, T. (2014). Delivering High Quality School-Wide Positive Behavior Support in Inclusive Schools. In J. McLeskey, N.L. Waldron, F. Spooner, and B. Algozzine (Eds.), Handbook of Effective Inclusive Schools (pp. 306-321). New York: Routledge.

Book chapter, edition specified

Bradshaw, C.P., Bottiani, J., Osher, D., Weissberg, R., and Sugai, G. (2014). The Integration of Positive Behavioral Interventions and Supports and Social and Emotional Learning. In M.D. Weist, N.A. Lever, C.P. Bradshaw, and J. Owens (Eds.), Handbook of School Mental Health: Advancing Practice and Research (2nd ed., pp. 101-118). New York: Springer.

Pas, E.T., Bradshaw, C.P., and Cash, A. (2014). Coaching Classroom-Based Preventive Interventions. In M. Weist, N. Lever, C. Bradshaw, and J. Owens (Eds.), Handbook of School Mental Health (2nd ed., pp. 255-267). New York: Springer.

Journal article, monograph, or newsletter

Bottiani, J.H., Bradshaw, C.P., Rosenberg, M.S., Hershfeldt, P.A., Pell, K.L., and Debnam, K.J. (2012). Applying Double Check to Response to Intervention: Culturally Responsive Practices for Students With Learning Disabilities. Insights on Learning Disabilities: From Prevailing Theories to Validated Practices, 9(1): 93-107.

Bradshaw, C.P. (2013). Preventing Bullying Through Positive Behavioral Interventions and Supports (PBIS): A Multitiered Approach to Prevention and Integration. Theory Into Practice, 52(4): 288-295.

Bradshaw, C.P. (2015). Translating Research to Practice in Bullying Prevention. American Psychologist, 70(4): 322-332.

Bradshaw, C.P., and Haynes, K.T. (2012). Building a Science of Partnership-Focused Research: Forging and Sustaining Partnerships to Support Child Mental Health Prevention and Services Research. Administration and Policy in Mental Health and Mental Health Services Research, 39(4): 221-224.

Bradshaw, C.P., and Pas, E.T. (2011). A State-Wide Scale-Up of School-Wide Positive Behavioral Interventions and Supports (PBIS): Developing Systems to Support and Assess Adoption, Implementation, and Outcomes. School Psychology Review, 40: 530-548.

Bradshaw, C.P., and Waasdorp, T. (2009). Measuring and Changing a "Culture of Bullying". School Psychology Review, 38(3): 356-361.

Bradshaw, C.P., Goldweber, A., Fishbein, D., and Greenberg, M.T. (2012). Infusing Developmental Neuroscience Into School-Based Preventive Interventions: Implications and Future Directions. Journal of Adolescent Health, 51(2): 41-47.

Bradshaw, C.P., Mitchell, M.M., O'Brennan, L.M., and Leaf, P.J. (2010). Multilevel Exploration of Factors Contributing to the Overrepresentation of Black Students in Office Disciplinary Referrals. Journal of Educational Psychology, 102(2): 508-520.

Bradshaw, C.P., Pas, E., Barrett, S., Bloom, J., Hershfeldt, P., Alexander, A., McKenna, M., and Leaf, P. (2012). A State-Wide Partnership to Promote Safe and Supportive Schools: The PBIS Maryland Initiative. Administration and Policy in Mental Health and Mental Health Services Research, 39(4): 225-237.

Bradshaw, C.P., Pas, E.T., Goldweber, A., Rosenberg, M., and Leaf, P. (2012). Integrating School-Wide Positive Behavioral Interventions and Supports With Tier 2 Coaching to Student Support Teams: The PBISplus Model. Advances in School Mental Health Promotion, 5(3): 177-193.

Bradshaw, C.P., Waasdorp, T.E, and Leaf, P.J. (2012). Effects of School-Wide Positive Behavioral Interventions and Supports on Child Behavior Problems. Pediatrics, 130(5): 1136-1145.

Bradshaw, C.P., Waasdorp, T.E., and Leaf, P.J. (2015). Examining Variation in the Impact of School-Wide Positive Behavioral Interventions and Supports: Findings From a Randomized Controlled Effectiveness Trial. Journal of Educational Psychology, 107(2): 367-385.

Cash, A., Bradshaw, C.P., and Leaf, P.J. (2015). Observations of Student Behavior in Nonclassroom SettingsA Multilevel Examination of Location, Density, and School Context. Journal of Early Adolescence, 35(5): 597-627.

Debnam, K., Pas, E.T., and Bradshaw, C.P. (2012). Secondary and Tertiary Support Systems in Schools Implementing School-Wide Positive Behavioral Interventions and Supports: A Preliminary Descriptive Analysis. Journal of Positive Behavior Interventions, 14(3): 142-152.

Debnam, K.J., Pas, E., and Bradshaw, C.P. (2013). Factors Influencing Staff Perceptions of Administrator Support for Tier 2 and 3 Interventions: A Multilevel Perspective. Journal of Emotional and Behavioral Disorders, 21(2): 116-126.

Domitrovich, C.E., Bradshaw, C.P., Greenberg, M.T., Embry, D., Poduska, J.M., and Ialongo, N.S. (2010). Integrated Models of School-Based Prevention: Logic and Theory. Psychology in the Schools, 47(1): 71-88.

Dong, N., Reinke, W.M., Herman, K.C., Bradshaw, C P., and Murray, D.W. (2016). Meaningful Effect Sizes, Intraclass Correlations, and Proportions of Variance Explained by Covariates for Planning Two-and Three-Level Cluster Randomized Trials of Social and Behavioral Outcomes. Evaluation Review, 40(4), 334-377.

Dunlap, G., Kincaid, D., Horner, R.H., Knoster, T., and Bradshaw, C.P. (2014). A Comment on the Term "Positive Behavior Support". Journal of Positive Behavior Interventions, 16(3): 133-136.

Duong, J., and Bradshaw, C.P. (2013). Using the Extended Parallel Process Model to Examine Teachers' Likelihood of Intervening in Bullying. Journal of School Health, 83(6): 422-429.

Goldweber, A., Waasdorp, T.E., and Bradshaw, C.P. (2013). Examining the Link Between Bullying Profiles and Perceptions of School Climate: A Latent Class Approach. Journal of School Psychology, 51(4): 469-485.

Hershfeldt, P., Rosenberg, M., and Bradshaw, C.P. (2011). Function-Based Thinking: A Systematic way of Thinking About Function and its Role in Changing Student Behavior Problems. Beyond Behavior, 19(3): 12-21.

Hershfeldt, P.A., Pell, K. Sechrest, R., Pas, E.T., and Bradshaw, C.P. (2012). Lessons Learned Coaching Teachers in Behavior Management: The PBIS "Plus" Coaching Model. Journal of Educational and Psychological Consultation, 22(4): 280-299.

Mitchell, M.M., Bradshaw, C.P., and Leaf, P.J. (2010). Student and Teacher Perceptions of School Climate: A Multilevel Exploration of Patterns of Discrepancy. Journal of School Health, 80(6): 271-279.

O'Brennan, L., Bradshaw, C.P., and Furlong, M. (2014). Influence of Classroom and School Climate on Teacher Perceptions of Student Problem Behavior. School Mental Health, 6(2): 125-136.

O'Brennan, L., Waasdorp, T.E., Pas, E.T., and Bradshaw, C.P. (in press). Impact of School Bullying on Subsequent Social-Emotional Functioning: A Comparison of Students in General and Special Education. Remedial and Special Education.

Pas, E., Bradshaw, C.P. and Mitchell, M.M. (2011). Examining the Validity of Office Discipline Referrals as an Indicator of Student Behavior Problems. Psychology in the School, 48(6): 541-555.

Pas, E.T., and Bradshaw, C.P. (2012). Examining the Association Between Implementation and Outcomes: State-Wide Scale-Up of School-Wide Positive Behavior Intervention and Supports. Journal of Behavioral Health Services and Research, 39(4): 417-433.

Pas, E.T., and Bradshaw, C.P. (2014). What Affects Teacher Ratings of Student Behaviors? The Potential Influence of Teachers' Perceptions of the School Environment and Experiences. Prevention Science, 15(6): 940-950.

Pas, E.T., Bradshaw, C.P., and Hershfeldt, P.A. (2012). Teacher- and School-Level Predictors of Teacher Efficacy and Burnout: Identifying Potential Areas for Support. Journal of School Psychology, 50(1): 129-145.

Pas, E.T., Bradshaw, C.P., Hershfeldt, P.A., and Leaf, P.J. (2010). A Multilevel Exploration of the Influence of Teacher Efficacy and Burnout on Response to Student Problem Behavior and School-Based Service Use. School Psychology Quarterly, 25(1): 13-27.

Pas, E.T., Waasdorp, T.E., and Bradshaw, C.P. (2014). Examining Contextual Influences on Classroom-Based Implementation of Positive Behavior Support Strategies: Findings From a Randomized Controlled Effectiveness Trial. Prevention Science, 16(8): 1096-106.

Racz, S.J., O'Brennan, L.M., Bradshaw, C.P., and Leaf, P.J. (2016). The Influence of Family and Teacher Factors on Early Disruptive School Behaviors: A Latent Profile Transition Analysis. Journal of Emotional and Behavioral Disorders, 24(2), 67-81.

Stuart, E.A., Bell, S.H., Ebnesajjad, C., Olsen, R.B., and Orr, L.L. (2017). Characteristics of School Districts That Participate in Rigorous National Educational Evaluations. Journal of Research on Educational Effectiveness, 10(1), 168-206.

Stuart, E., Cole, S., Bradshaw, C.P., and Leaf, P.J. (2011). The Use of Propensity Scores to Assess the Generalizability of Results From Randomized Trials. The Journal of the Royal Statistical Society, Series A, 174(2): 369-386.

Stuart, E.A., Bradshaw, C.P., and Leaf, P.J. (2015). Assessing the Generalizability of Randomized Trial Results to Target Populations. Prevention Science, 16(3): 475-485.

Sullivan, T., and Bradshaw, C.P. (2012). Serving the Needs of Youth With Disabilities Through School-Based Violence Prevention Efforts. Behavioral Disorders, 37(3): 129-132.

Swanson, H.L. (2012). Cognitive Profile of Adolescents With Math Disabilities: Are the Profiles Different From Those With Reading Disabilities?. Child Neuropsychology, 18(2): 125-143.

Waasdorp, T.E., Bradshaw, C.P., and Leaf, P.J. (2012). The Impact of Schoolwide Positive Behavioral Interventions and Supports on Bullying and Peer Rejection: A Randomized Controlled Effectiveness Trial. Archives of Pediatrics and Adolescent Medicine, 116(2): 149-156.

Supplemental information

Co-Principal Investigators: Phillip Leaf and Nicholas Ialongo

Questions about this project?

To answer additional questions about this project or provide feedback, please contact the program officer.

 

Questions about this project?

To answer additional questions about this project or provide feedback, please contact the program officer.

 

You may also like

Zoomed in IES logo
Contract

What Works Clearinghouse Supporting and Analyzing ...

Contract number: 91990025F0014
Read More
Zoomed in IES logo
Training Material

Reduced calendar years: How are they implemented, ...

Read More
Zoomed in IES logo
Fact Sheet/Infographic/FAQ

Consistent Implementation of a Multi-Tiered System...

Read More