Co-Principal Investigator: Algozzine, Bob

Purpose: The goal of this project was to examine the efficacy of Team-Initiated Problem Solving (TIPS), a model for training and coaching school teams to use behavioral and academic data to define and solve problems. Schoolwide Positive Behavioral Interventions and Supports (PBIS) is a frequently used systems-level intervention that involves school teams actively engaging problem solving to improve student academic, social, emotional, and behavioral outcomes. This requires both appropriate data management systems and practical procedures for using data in daily problem solving. To meet this need, in a prior IES project, TIPS was developed and demonstrated feasibility of implementation by school teams as well as promise for improving team problem solving. Building upon the prior study, this project conducted a randomized controlled trial to examine the efficacy of TIPS for not only improving team problem solving, but also improving team implementation of solutions and student academic and behavioral outcomes.

Project Activities: In Phase 1, researchers revised, updated, and evaluated a measure to assess if team-generated solutions are implemented as planned. During Phase 2, teams were recruited and randomly assigned to either the immediate TIPS intervention condition or a wait-list control condition. Multiple measures were used to assess the impact of TIPS on school team and student outcomes. In Phase 3, researchers revised the TIPS training materials for dissemination by state and district trainers.

WWC Review: This study (Horner et al., 2018) was reviewed and met WWC group design standards without reservations.

Key Outcomes: The main findings from this project are as follows:

• As a result of Phase 1 activities, the Decision Observation, Recording, and Analysis (DORA) tool, a measure of team-based problem solving, was revised to also assess the extent to which team-generated solutions were implemented with fidelity. Scores on the resulting measure, the DORA-II were found to be valid indicators of problem solving and provided technically adequate information on the extent to which teams had implemented a solution and whether it made a difference for students (Algozzine et al., 2016).
• Results from the Phase 2 efficacy study indicated that teams who participated in TIPS demonstrated greater improvements in problem-solving, decision-making, and meeting outcomes compared to teams in the wait-list control group. In addition, schools with teams that participated in TIPS were more likely to implement solutions developed via the TIPS problem-solving protocols and had fewer out-of-school suspensions than schools in the control group (Horner et al., 2018).

Products: The products of this project included evidence of the efficacy of TIPS, published reports, and presentations.

Structured Abstract

Setting: The research took place in elementary schools in Oregon and North Carolina.

Sample: A total of 10 school-based teams were involved in Phase 1 and 38 in Phase 2.

Intervention: TIPS includes access to academic and behavioral referral data from the School-Wide Information System, a 6-hour professional development workshop for the school team, and two post-training coached team meetings. The TIPS training involves orientation to core problem-solving foundations and practices, including (1) the roles, scheduling, and recording format for an effective meeting; (2) the process for defining academic and/or behavior problems with precision; (3) the development of action plans that draw from existing research yet tailor practices to fit the local context; and (4) the implementation, assessment, and adaptation of action-planning solutions. The logic guiding all elements of TIPS is that problem solving that includes objective data about the context, behavior, and outcomes — coupled with formal procedures for using data for problem solving — will result in solutions that are more likely to be implemented and more likely to benefit students socially and academically.

Research Design and Methods: Phase 1 focused on revising the DORA to assess the extent to which team-generated solutions are implemented. The revised measure was validated by observing 10 school teams and comparing the resulting scores to data from permanent products that would result from solution implementation or direct observations of whether a solution was implemented. During Phase 2, researchers conducted a randomized controlled trial with school teams as the unit of randomization. Teams were randomly assigned to TIPS or to a services-as-usual wait-list control. The immediate and short-term (6 months following TIPS training) effects of TIPS on teams and students were evaluated using multiple measures. In Phase 3, researchers revised the TIPS training materials for dissemination by state and district trainers.

Control Condition: Teams in the wait-list comparison group continued using their typical procedures.

Key Measures: The DORA-II was used to measure the fidelity with which school teams implemented the TIPS problem-solving processes as well as team-generated solutions. School team staff also rated the perceived impact of each solution on student outcomes. Student behavioral outcomes were assessed by examining office discipline referral data and student academic outcomes were assessed using the Dynamic Indicators of Basic Early Literacy Skills (DIBELS). The research team also collected data related to specific behavioral/academic goals (such as attendance or assignment completion).

Data Analytic Strategy: Descriptive summaries and inferential statistics (t-test, analysis of variance) were used to describe and compare pretest, posttest, and follow-up scores for all dependent variables (DORA-II scores, academic and social/behavioral indicators) across treatment and control groups. Repeated measures analysis of variance was used to assess the impact of TIPS on teams' use of problem-solving processes, improvement in teams' implementation of their selected solution actions, and resulting impacts on student academic and behavioral outcomes. Cohen's d (standardized mean difference effect size) and effect size confidence intervals were reported for all dependent measures.

Related IES Projects: Enhancing Data-based Decision-Making in Schools (R324A070226); TIPS EdTech: Developing Professional Development and Online Applications to Support Team-Initiated Problem Solving (TIPS) within Multi-Tiered Support Systems (R324A170052)

Additional Online Resources and Information: TIPS materials and videos can be accessed at https://www.pbis.org/resource-type/materials#tips and https://www.pbis.org/video-examples/video#tips.

Products and Publications

Book chapter

Newton, J.S., Todd, A.W., Algozzine, B., Algozzine, K., Horner, R.H., and Cusumano, D.L. (2014). Supporting Team Problem Solving in Inclusive Schools. In J. McLeskey, N.L. Waldron, F. Spooner, and B. Algozzine (Eds.), Handbook of Effective Inclusive Schools: Research and Practice (pp. 275–291). New York: Routledge.

Book chapter, edition specified

Todd, A., Algozzine, B., Horner, R., and Algozzine K. (in press). Data-Based Decision-Making. In C. Reynolds, K. Vannest, and E. Fletcher-Janzen (Eds.), Encyclopedia of Special Education: A Reference for the Education of Children, Adolescents, and Adults With Disabilities and Other Exceptional Individuals (4th ed.). Hoboken, NJ: John Wiley and Sons.

Journal article, monograph, or newsletter

Algozzine, B., Horner, R.H., Todd, A.W., Newton, J.S., Algozzine, K., and Cusumano, D. (2016). Measuring the Process and Outcomes of Team Problem Solving. Journal of Psychoeducational Assessment, 34(3): 211–229. doi:10.1177/0734282915592535

Horner, R. H., Newton, J. S., Todd, A. W., Algozzine, B., Algozzine, K., Cusumano, D., & Preston, A. (2018). A Randomized Waitlist Controlled Analysis of Team-Initiated Problem Solving Professional Development and Use. Behavioral Disorders, 43(4): 444–456. doi:10.1177/0198742917745638

Todd, A. W., Algozzine, B., Horner, R. H., Preston, A. I., Cusumano, D., & Algozzine, K. (2019). A Descriptive Study of School-Based Problem-Solving. Journal of Emotional and Behavioral Disorders, 27 (1): 14–24. doi:10.1177/1063426617733717