Co-Principal Investigator: Alexander, Andrea (Maryland State Department of Education); Pas, Elise (Johns Hopkins University)

Purpose: In this project, researchers will evaluate the impacts of Positive Behavioral Interventions and Supports (PBIS), a multi-tiered intervention to prevent student behavior problems and promote a positive school environment to support academic success. The Maryland State Department of Education has built the infrastructure for statewide implementation of PBIS through a long-standing partnership with Johns Hopkins University and Sheppard Pratt Health System. Maryland has provided training to almost 1000 schools on implementing the first tier (school-wide components) of PBIS, has mandated the implementation of PBIS (or an alternative program) in middle and high schools that exceed a truancy threshold, and has begun evaluating an enhanced version for high schools that improves tier 1 and includes tier 2 components (aimed at students who do not respond to the tier 1 components). The State is now in a position to learn whether its past efforts have led to beneficial student outcomes and to consider its plans for wider implementation and implementation of a more intensive version of PBIS.

Project Activities: The research team will carry out three evaluation studies of PBIS: (1) a retrospective quasi-experimental study comparing elementary schools that adopted single-tier PBIS with those that did not, (2) a retrospective quasi-experimental study of middle and high schools that voluntarily adopted single-tier PBIS versus those that were mandated to adopt it, and (3) a randomized control trial comparing elementary and middle schools using a single-tier version of PBIS versus those using a modified two-tiered version (less intensive and expensive than the one being evaluated in high schools). In addition, researchers will determine the costs of implementing both the single and modified two-tiered versions of PBIS.

Products: The products of this project include evidence of the efficacy of single-tier PBIS, of mandating its adoption, and of any added benefit of implementing the two-tiered version. The evidence will be provided directly to the Maryland State Department of Education through the participation of department personnel on the project and through annual briefings. Researchers will present results to the research and practice community through presentations, peer-reviewed publications, and postings on state and national websites.

Structured Abstract

Setting: This project will take place in the state of Maryland.

Sample: The first evaluation study will draw on the entire population of 866 elementary schools in Maryland, of which 456 have been trained in single-tier PBIS with the remainder to provide a source for the matched comparison group. The second study will include the middle and high schools mandated to implement PBIS (104 schools with 21 middle and 83 high schools) and schools with similar truancy rates that had earlier volunteered to implement PBIS. The third study will use a volunteer sample of about 80-85 elementary and middle schools from across the state.

Intervention: PBIS includes a set of school-wide (tier 1) components that are to reduce student problem behavior and create a positive school climate. These include (1) establishing positive behavior expectations and rewarding such behavior, (2) providing explicit classroom instruction to students on PBIS, (3) establishing and enforcing a formal behavior violation system, and (4) collecting and using data to identify higher-need students who require further activities (that may include parental involvement). The implementation of these components is supported at the school level by a PBIS team and a PBIS coach who provides training to administration, staff, and the team as well as at the state level by a team that coordinates state and district efforts and provides training for coaches and school teams. The modified version of PBIS, which will be evaluated in the third study, additionally includes participation in a school-level climate survey, training in the use of the survey data, training in identifying students and subgroups who need additional support, and training in implementing programs to support those students (tier 2 components).

Research Design and Methods: The first study is based on retrospective data from 2008–2014. It will match the 456 elementary schools trained in single-tier PBIS to a comparison group of schools that were not trained (propensity score matching will use such variables as student composition, attendance/truancy, suspensions and, student academic achievement). The second retrospective study will use propensity score matching (using the same matching variables as the first study) to compare student outcomes in 104 middle and high schools that were mandated to adopt single-tier PBIS (from 2010–2014) against matched middle and high schools that volunteered to adopt single-tier PBIS (from 2008–2010). The third study is a RCT that will randomly assign 80 to 85 elementary and middle schools to either receive training in single-tier PBIS or the modified two-tier PBIS and compare the two groups' student education outcomes.

The cost analysis will obtain costs for single-tier PBIS from a random sample of about 200 schools stratified on school level, enrollment, district, and free and reduced lunch participation and the costs for the modified two-tier PBIS will be obtained from 15 to 20 schools taking part in Study 3.

Control Condition: For the first retrospective study, students in the comparison group are not expected to have been exposed to PBIS components. For the second retrospective study, students in both groups are expected to have been exposed to single-tier PBIS (the difference being voluntary versus mandated implementation). For the third study, students in the control group will have exposure to single-tier PBIS (through voluntary implementation).

Key Measures: For the two retrospective studies, the outcomes are suspensions, attendance, truancy, and scores on the Maryland School Assessment in reading and math (using both scores and proficiency levels). For the third study, the outcomes are suspensions, attendance, truancy, special education referrals and placements, achievement on the Partnership for Assessment of Readiness for College and Careers (PARCC) assessment, and student survey responses concerning engagement, school environment, and school safety using the Maryland school climate survey. In addition, fidelity of PBIS implementation will be examined using three instruments: the Implementation Phases Inventory, School-wide Evaluation Tool, and the Benchmarks of Quality).

Data Analytic Strategy: For the two retrospective studies, researchers will estimate propensity scores using logistic regression to predict whether a school was exposed to PBIS (for study 1) or volunteered for PBIS (for study 2) based on student composition, suspension, truancy, and achievement data in order to identify comparison schools. For all three studies, researchers will use multi-level models to estimate the effects of PBIS on the outcomes. The research team will use two-level models to examine students in schools and three-level models will be used to examine within-student repeated measures. In addition, researchers will use moderator analyses to examine fidelity of implementation, school demographics, and school baseline climate.

Products and Publications

Journal article, monograph, or newsletter

O'Brennan, L., Pas, E., and Bradshaw, C. (2017). Multilevel Examination of Burnout Among High School Staff: Importance of Staff and School Factors. School Psychology Review, 46(2), 165–176.

Sheras, P.L., and Bradshaw, C.P. (2016). Fostering Policies That Enhance Positive School Environment. Theory Into Practice, 55(2), 129–135.