Co-Principal Investigators: Jon Jagemann (Milwaukee Public Schools), Evan Hart (University of Southern Mississippi)

Partners: University of Wisconsin-Milwaukee, Milwaukee Public Schools (MPS)

Purpose: One approach to increasing student engagement and achievement is through the reduction of student disruptive behavior and positive reinforcement of behaviors that support academic success. Under the framework of School-Wide Positive Behavior Intervention and Supports (SWPBIS), schools use behavior analysis and positive reinforcement in a tiered system that increases in intensity for students who do not respond to the previous tier. Milwaukee Public Schools (MPS) uses a three-tiered system in which the intervention Check-in/Check Out (CICO) is its Tier 2 intervention. MPS is concerned that CICO—which is based on adult mentoring—may not be enough to improve certain disruptive behaviors. This project will evaluate a modified version of CICO that involves functional behavior assessment and reinforcement to determine if it leads to reduced referrals to Tier 3 and other desired behavioral and academic outcomes.

Project Activities: Researchers in the partnership will conduct a randomized controlled trial in which 50 schools are randomly assigned to either use the traditional version of CICO or the modified version. Researchers will compare the behavioral outcomes of students referred to a Tier 2 intervention at the treatment versus the control schools.

Products: The evaluation will provide evidence of the efficacy of a modified Tier 2 behavioral intervention, CICO. The evidence will be directly provided to MPS through participation of department personnel on the project. The findings will be shared through briefings with MPS officials and in an online brief. The partners also plan to make presentations to academic and practitioner audiences and to publish in journals.

Structured Abstract

Setting: This project will take place in Milwaukee Public Schools in Wisconsin.

Population/Sample: The study will include schools that serve a K–5 or K–8 grade range and that met Tier 1 fidelity during the 2015–2016 school year (83 schools in 2014–2015). The partnership will randomly select 50 of the eligible schools to take part in the study. Eligible students include those who do not respond to Tier 1 behavior supports (approximately 1,500 students, or 10% of all enrollees).

Intervention: Milwaukee Public Schools (MPS) uses a three-tiered system to provide behavioral supports to students in order to reduce problem behaviors and positively reinforce appropriate barriers in order to increase student engagement and achievement. The Tier 1 intervention reinforces positive behaviors by allowing students to earn tokens for displaying reinforced behaviors which can be exchanged for goods and services. Students who display undesired behavior (as identified by teachers, parents, discipline referrals, or suspensions) are referred to a Building Intervention Team which determines if they will receive CICO, the Tier 2 intervention.

The standard version CICO involves assigned adult mentors who meet with students at the beginning and end of each school day to go over expected behavior and review behavior ratings from their teachers. If the ratings show that students met their behavioral goal for the day, students receive verbal praise and/or rewards; if not, the mentors offer advice for improving the next day. Under the modified version of CICO, the mentors receive training in assessing students using a functional behavior assessment and identifying a specific functional reinforcement from a standardized list that should reinforce the desired behavior when the student has displayed it. For example, if the mentor identifies a student's desire for peer attention as a cause of undesired behavior, the mentor can offer extra time with a preferred peer as a reinforcer when the student displays the desired behavior.

Students who do not meet a performance threshold in Tier 2 for four weeks (and students who are directly referred to Tier 3 by a parent or school staff) are reviewed by the Building Intervention Team to determine their referral to Tier 3. If referred, Tier 3 facilitators determine the intervention to be used with the student/family. The Tier 3 referral process is the same in the treatment and comparison schools.

Research Design and Methods: The partnership team will randomly select 50 schools from the eligible population of schools (schools with a K–5 or K–8 grade configuration that met Tier 1 fidelity in 2015–2016). They will match these 50 schools into 25 pairs based on 2015–2016 data including daily progress reports, office discipline referrals, Tier 2 fidelity, and the number of students in grades K–5 that receive CICO. Members of each pair will be randomly assigned to either the standard CICO or the modified CICO Tier 2 intervention condition.

Control Condition: Students in the control group will receive the standard CICO Tier 2 intervention.

Key Measures: The evaluation will use two outcome measures: Daily Progress Reports and Office Discipline Referrals. For Daily Progress Reports (DPR), students receive a teacher rating on 9 defined behaviors three-to-five times a day. The evaluation will use the average percentage of DPR points earned each day out of the maximum possible as an outcome variable. The second outcome variable will be office discipline referrals, when students are referred to the office for behavioral infractions while in school. (Most students referred to Tier 2 have received one referral. Two or more referrals will be considered as a higher-risk status.) The partnership team will examine the fidelity of the standard and the modified version of the CICO Tier 2 intervention using a 12-item measure of the core components, and will introduce additional questions and conduct a review to determine if appropriate treatment was provided to the students in the modified version.

Data Analytic Strategy: The evaluation will use a two-level HLM model (students in schools) to analyze the outcome data. Because office discipline referrals is a binary variable (less than 2 versus 2 or more), researchers will use a hierarchical generalized linear model to predict the probability of falling into the desired category. The evaluation will also include an analysis of academic outcomes such as grades. The researchers will include student-level covariates that were significant in previous studies of SWPBIS (e.g., male, Black, Free and Reduced Lunch eligibility), and will investigate school-level covariates for which there is divergent evidence (e.g., percentage minority, schoolwide achievement, schoolwide poverty).Two fidelity analyses will be done. First, fidelity of the additional aspects of functional CICO (as added to the standard CICO) will be examined to see if they can be implemented with fidelity. Second, researchers will examine if fidelity of the standard 2 CICO model is similar for the treatment and control groups. If not, they will examine differences in fidelity as a covariate.