Skip Navigation
Funding Opportunities | Search Funded Research Grants and Contracts

IES Grant

Title: Long-Term Follow-Up of a PBIS RCT: Impact on Academics, Behavior, and Juvenile Justice Involvement
Center: NCER Year: 2023
Principal Investigator: Henneberger, Angela Awardee: University of Maryland, Baltimore
Program: Social, Emotional, and Behavioral Context for Teaching and Learning      [Program Details]
Award Period: 3 years (07/01/2023 – 06/30/2026) Award Amount: $1,456,257
Type: Follow-Up Award Number: R305A230398
Description:

Co-Principal Investigators: Bradshaw, Catherine; Pas, Elise; Goldstein, Ross

Purpose: In this 16-year long-term follow-up study of an IES-funded efficacy trial, the researchers will examine the main, moderated, and generalized effects, as well as the long-term cost-benefits of the multi-tiered Positive Behavioral Interventions and Supports (PBIS) framework as implemented in Maryland. PBIS is a widely disseminated prevention model in the United States that uses a public health approach by providing universal foundational behavioral supports to all students to effectively teach and promote positive student behavior (Tier 1) with the provision of targeted (Tier 2) and intensive (Tier 3) supports for those not responding to Tier 1. The primary aim of the 2007 initial efficacy trial was to determine whether students in elementary schools randomized to implement PBISplus (PBIS Tiers 1 + 2) had lower rates of special education evaluation referrals, disciplinary referrals, and better academic performance than students in schools implementing the standard Maryland PBIS Tier 1 only model. The study found evidence of positive effects in the short term, including improvements in rates of special education service use and teacher-reported student academic performance and teachers' efficacy. The PBISplus trial together with this research team's unique access to statewide data across multiple service sectors through the Maryland Longitudinal Data System (MLDS) Center provide the ideal context to fill a major research gap. This study also addresses timely and substantial practice needs, as nearly all states in the U.S. are attempting to scale-up multi-tiered PBIS.

Project Activities: This project has four specific aims:

  1. Determine the long-term impact of the Tier 1 + 2 supports relative to Tier 1 only on middle and high school behavioral and academic outcomes such as disruptive behavior and academic achievement; juvenile justice outcomes such as arrests and detainment; postsecondary outcomes such as college enrollment, persistence, and degree attainment; and early labor outcomes such as workforce participation and early labor market earnings.
  2. Determine whether the long-term impacts are moderated by student risk based on baseline teacher behavioral rating scores and baseline academic achievement or whether Tiers 1 + 2 are most effective (or close gaps) for particular student groups including students receiving special education services, students of color, disadvantaged students, and English language learners.
  3. Examine the generalizability of program impacts by estimating these Tiers 1 + 2 effects in the population of students and schools in the state of Maryland.
  4. Determine the long-term cost-benefit of Tiers 1 + 2, including the return on investment of the impact across the education, juvenile justice, and workforce sectors.

Products: The research team will disseminate the main, moderated, generalized, and cost effects of PBIS Tiers 1 + 2 through peer-reviewed manuscripts, presentations, cost trainings, and free online research reports/briefs working in close collaboration with practice partners and stakeholders from several Maryland agencies and organizations including http://www.pbismaryland.org/resources/research-briefs and http://mldscenter.maryalnd.gov/CenterOutput.html).

Structured Abstract

Setting: The original PBISplus trial included 42 Maryland public, comprehensive elementary schools. All schools in the trial were previously trained to implement PBIS Tier 1 by the PBIS Maryland State Team (MSDE and Sheppard Pratt Health System). In this follow-up study, the researchers will link trial data with archival data from all students and schools in Maryland through the MLDS in order to conduct a long-term follow-up, draw propensity score matched schools for exploratory quasi-experimental contrasts with non-PBIS schools, conduct generalizability analyses, and perform a cost-benefit analysis across multiple service sectors.

Sample: In the previous randomized control trial (RCT) study, 29,569 students and from 3,202 staff participated. The student sample were majority male (52 percent) and not White (54 percent African American, 8 percent Hispanic of any race, and 33 percent White). Most staff were female (91 percent) and White (75 percent). The schools served a sizable proportion of students who received free and reduced meals (43 percent) and were highly mobile.

Intervention: The original PBISplus RCT examined the combined impacts of Tier 1 + 2 PBIS supports as compared to Tier 1 only. PBISplus provided training and coaching in Tier 2 supports, including function-based thinking, a simplified functional behavioral analysis model, Check-In/Check-Out, and culturally responsive practices to reduce disproportionality in discipline and special education referrals.

Research Design and Methods: The initial efficacy trial examined the efficacy of the PBIS Tiers 1 + 2 relative to Tier 1 only, using a group RCT design. The researchers randomized elementary schools to receive either PBISplus (Tiers 1 + 2; intervention; n = 22 schools) or continued participation in the statewide standard PBIS Tier 1 programing (comparison; n = 20). The researchers collected four waves of data (baseline and three spring timepoints) in the original RCT. In this follow-up study, the research team will merge data from the RCT with MLDS data across multiple service sectors spanning the years 2007-08 through 2022-23 to support this 16-year follow-up study.

Control Condition: The comparison schools only received training in PBIS Tier 1 supports.

Key Measures: RCT data include teacher-ratings of students' behavior, engagement ,and social emotional functioning; academic performance; staff self-reports of efficacy, behavior management, and school climate; and fidelity of implementation. MLDS data include standardized test scores, high school completion, educational service utilization, discipline problems and arrests, postsecondary readiness, postsecondary entry and degree attainment, and labor market outcomes.

Data Analytic Strategy: The researchers will leverage intent-to-treat and moderated main-effects models using multi-level modeling and multilevel growth models to examine primary aims. They will use complier average causal effect estimation and propensity scores for a quasi-experimental exploratory sub aim. They will use propensity scores in a separate generalizability analysis.

Cost Analysis: The research team will use the ingredients method to enumerate the costs of the PBISplus intervention based on information derived from fidelity data and coaching log data, taking into consideration opportunity costs. They will also conduct a cost-benefits analysis using a shadow pricing approach to estimate the cost savings of PBISplus in relation to any long-term benefits identified across multiple service sectors (education, labor market, justice system).

Related IES Projects: Testing The Impact of PBIS Plus (R324A070118), Examining Variation in the Impact of School-Wide Positive Behavioral Interventions and Supports (PBIS)(R305A090307), Double Check: A Cultural Proficiency and Student Engagement Model(R324A110107), Evaluating Maryland State Policies to Improve School Climate (R305H150027), Testing the Efficacy of Double Check: A Cultural Proficiency Professional Development Model in Middle Schools (R305A150221), The National Center for Rural School Mental Health (NCRSMH): Enhancing the Capacity of Rural Schools to Identify, Prevent, and Intervene in Youth Mental Health Concerns (R305C190014)


Back