Project Activities
The researchers developed the PACT using expert feedback and iterative refinements. They tested usability and feasibility of the PACT in elementary schools that were actively preparing for or currently implementing a common multi-tiered system of support, School-wide Positive Behavior Interventions and Support (SWPBIS). The researchers evaluated the convergent, discriminant, and predictive validity of the PACT for SWPBIS and Check-In Check-Out (CICO) implementation, a common tier 2 intervention used in the SWPBIS framework. The researchers also evaluated the usability of PACT data reports to inform strategic implementation support. Finally, they determined the costs associated with administering, interpreting, and using the PACT.
Structured Abstract
Setting
This study took place in schools in Minnesota and Michigan that were actively preparing to or currently implementing School-wide Positive Behavior Interventions and Supports (SWPBIS) with a specific focus on tier 1 Positive Behavior Interventions and Supports (PBIS) and tier 2 Check-In Check-Out (CICO).
Sample
Across multiple studies, participants included approximately 700 staff from 332 schools that were sampled to be demographically representative of urban and rural schools.
The PACT captures the commitment of school-level staff to implement core practice elements of evidence-based practices that support positive social, emotional, and behavioral outcomes for students. The PACT’s assessment framework is informed by the theory of planned behavior that posits attitudes, subjective norms, and perceived behavioral control are key mechanisms linked to behavior change through commitment to implementation. Schools can use the PACT to provide school staff with data to inform tailoring of implementation strategies as part of data-driven continuous improvement cycles across all stages of the implementation process (exploration, preparation, implementation, sustainment) to maximize fidelity and impact.
Research design and methods
The researchers conducted five studies. In the first study, they used the Delphi Method, a defined process for reaching consensus among experts, to identify and define practice elements. They used these elements to develop the item pool for customizable referents. In the second study, they constructed the PACT with feedback from experts on item coverage, breadth of practice elements within the item pool, feasibility, and usability. In the third and fourth studies, the researchers collected data from educators at multiple time points to examine the PACT's psychometric properties including convergent, discriminant, and predictive validity. In the final study, they evaluated the usability of PACT data reports to inform strategic tailoring of implementation supports.
Control condition
There was no control condition for this study.
Key measures
The researchers assessed educators' self-efficacy, social norms, professional identity, and attitudes toward evidence-based practices. They assessed the acceptability and appropriateness of PACT, educator implementation of tier 1 PBIS or tier 2 CICO, and classroom-level behavioral outcomes.
Data analytic strategy
The researchers analyzed expert and practitioner feedback within a consensus building process that combined qualitative and quantitative information. They determined the structural validity of the PACT by examining response distributions as well as item skewness and kurtosis. In addition, the research team assessed internal consistency of the PACT using Cronbach's coefficient alpha and inter-item correlations. They conducted confirmatory factor analyses, analyses of measurement invariance, and structural equation modeling to examine the psychometric properties of the PACT.
Cost analysis strategy
The researchers determined the costs of administering, interpreting, and using the PACT using standard average weighted cost metrics. They considered inputs such as time, supplies, overhead, and other costs associated with PACT such as scheduling PACT administration and attending meetings.
Key outcomes
Key findings will be reported later once peer reviewed publications describing the key outcomes of the funded project are available.
People and institutions involved
IES program contact(s)
Products and publications
Products: Products include the Planned Action and Commitment Tool (PACT) for practitioners and researchers who support education systems at any stage of implementation of an evidence-based practice (EBP) to address the social, emotional, or behavioral needs of students. Dissemination products include detailed descriptions of the PACT, concise summative feedback reports to each participating school district and the Minnesota Department of Education, and peer reviewed publications and presentations at a wide variety of research and practice conferences.
Publications:
ERIC Citations: Find available citations in ERIC for this award here.
Select Publications:
Filter, K.J., Ford, A.L.B., Bullard, S., Cook, C.R., Sowle, C.A., Johnson, L.D., Kloos, E., & Dupuis, D. (2022). Distilling Check-in/Check-Out into Its Core Practice Elements Through an Expert Consensus Process. School Mental Health, 14, 695-708. doi.org/10.1007/s12310-021-09495-x
Filter, K.J., Johnson, L.D., Ford, A.L.B., Sowle, C.A., Bullard, S.J., Cook, C.R., Kloos, E., & Dupuis, D. (2022). An expert consensus process to distill tier 1 PBIS into core practice elements essential to improving front-line implementation. Education and Treatment of Children, 45, 51-67. doi.org/10.1007/s43494-021-00066-y
Supplemental information
Co-Principal Investigator: Filter, Kevin
Questions about this project?
To answer additional questions about this project or provide feedback, please contact the program officer.
