Project Activities
The research team will adapt and pilot test the team-based problem-solving intervention over the course of 4 years. The research team will begin with an initial adaptation of Outcomes: PME and measurement protocols based on expert feedback, observations, and focus groups. The team will then implement the initial iteration to examine its fidelity, feasibility and usability using AB single-case design studies, followed by a focus group of PST members to inform further revisions. A second iteration of Outcomes: PME will be tested using the same process and the researchers will assess the extent to which PSTs involved in the previous implementation continue to use the problem-solving protocol with fidelity. The final iteration of Outcomes: PME will be pilot tested using a multiple baseline single-case design study to determine its promise for improving team-based problem solving, teacher fidelity of EBPs, and student behavioral outcomes. The extent to which the intervention was sustained by PSTs participating in the earlier studies will also be evaluated.
Structured Abstract
Setting
The research will take place in elementary schools in Wisconsin.
Sample
The primary sample will be PSTs who demonstrate low frequency of PST meetings, low intervention implementation integrity, and low overall team functioning (based on interviews and data from PST meetings). During Year 1, four to six PSTs will participate. PST members (at least four members per team) include administrators, school psychologists, general and special education teachers and other school personnel. Each year, students who are referred to the PSTs for behavior problems and their teachers will participate. Six new PSTs will be recruited in each of Years 2 and 3 along with a subset of students (approximately 200 per year) referred to the PST and their teachers. In Year 4, seven new PSTs and referred students (approximately 200) and their teachers will participate. Across all years of the project, the research team will seek feedback from three researcher consultants and an advisory board comprised of five researchers with expertise in team-based problem solving, disproportionality, single-case design, and behavioral assessment.
Intervention
The project will result in an adapted version of Outcomes: PME for PSTs who struggle to meet regularly and function effectively. The Outcomes: PME problem-solving protocolwas originally developed as a one-on-one consultation model. The basic framework involves the use of data to inform decision making. Specifically, the Outcomes: PME problem-solving protocolfollows five steps: (1) identify the concern, describe the context, and establish baseline; (2) set meaningful goal(s) and benchmarks; (3) plan the intervention and specify progress-monitoring procedures; (4) monitor progress and analyze data; and (5) evaluate intervention outcomes and plan next steps. In the current project, this protocol will be adapted for PSTs to use in recommending EBPs for teachers to implement with students who are referred to the PST because of behavior issues. A train-the-trainer model will be developed such that one member of the PST will serve as a coach who will train and mentor the other team members on the problem-solving protocol. The final adapted intervention will include training for the coach on the problem-solving process and a manualized problem-solving protocol.
Research design and methods
The research will be conducted across 4 years. In Year 1, the research team will complete the initial adaptation of Outcomes: PME and measures based on feedback from consultants, advisory board members, and PSTs as well as observations of PST meetings and classroom interventions. Specifically, six PSTs will be recruited and their meetings observed and transcribed to establish baseline functioning and problem-solving processes. Classroom observations will also be conducted on students referred to the PST to inform revisions to student and teacher measures. Two focus groups will also be conducted with PSTs—one in the fall to better understand current practices and barriers to problem solving and use of evidence-based practices; and one in the spring to gather input on the initial iteration of Outcomes: PME. In Year 2, the research team will recruit six new PSTs and test the initial iteration of Outcomes: PME using AB single-case design studies to document the feasibility and fidelity of the coaching and use of the problem-solving protocol, teachers' implementation of EBPs, and changes in student problem behaviors. A focus group will be conducted in the spring of Year 2 to solicit feedback for revising the protocol for the following year. In Year 3, a second iteration of Outcomes: PME will be tested using the same process as Year 2 with a new set of PSTs. Researchers will also assess the extent to which fidelity of the coaching and problem-solving protocol is maintained among Year 2 PSTs. In Year 4, the research team will conduct a pilot study to test the promise of the final iteration of Outcomes: PME for improving team-based problem solving, teacher fidelity of EBPs, and student behavioral outcomes. A multiple baseline single-case design study will be used, with seven PSTs randomly assigned to baselines and intervention start times. The extent to which the intervention was sustained by Year 2 and 3 PSTs will also be evaluated in a follow-up assessment. Finally, reports will be developed for each PST and findings will be shared at conferences and in peer-reviewed publications as well as with the Wisconsin Department of Public Instruction.
Control condition
In the pilot study, the PSTs' business-as-usual functioning in the baseline phase will serve as the control condition.
Key measures
Measures will assess intervention fidelity, feasibility, and acceptability as well as team-, teacher-, and student-level outcomes. Process measures will include adapted versions of the Problem-Solving Intervention Acceptability Questionnaire (to assess PST members' perceptions of acceptability), the Student Plan and Outcomes Report (to assess the integrity of individual PST meetings), and checklists to determine the fidelity of coach training and implementation of the problem-solving protocol. Several measures will also be developed by the research team, including a feasibility measure and assessments to evaluate generalization and maintenance of procedures when researcher support is removed. To assess team problem solving the following measures will be used: Decision Observation, Recording, and Analysis II Tool (direct observation tool to assess school team meeting processes and decision making), Performance Diagnostic Checklist-Schools (self-report measure to assess PST members' perceptions of the problem-solving process), and PST Self-Assessment Survey (self-report measure to evaluate the degree to which a PST implements problem-solving procedures, PST member knowledge of and familiarity with problem-solving components, and the ability of the PST coach to implement essential problem-solving components). Two observational measures, Direct Behavior Ratings and the Functional Assessment Observation form will be used to measure student behavior and its possible functions, antecedents, and consequences, as well as teachers' fidelity of implementing EBPs suggested by the PST. Finally, to compare the PSTs' discussions of student behaviors to the direct classroom observations, the research team will use the Student Outcomes Rubric.
Data analytic strategy
For all years, qualitative data from focus groups will be analyzed for themes using NVivo software. In Year 1, descriptive analyses will be conducted to summarize data regarding team functioning and student behavior. For the Year 2 and 3 AB single-case design studies, visual analyses will be conducted to explore changes in team-based problem solving and teacher and student behavior. To analyze data from the Year 4 pilot study, researchers will conduct visual as well as statistical analyses, including randomization tests and effect size calculations. To determine the costs of the intervention, the research team will utilize the ingredients method.
People and institutions involved
IES program contact(s)
Products and publications
Products: This project will result in an adapted version of Outcomes: PME to improve PSTs' problem solving, teachers' implementation of EBPs, and student outcomes. The project will result in peer-reviewed publications and presentations as well as additional dissemination products that reach education stakeholders such as practitioners and policymakers.
Supplemental information
Co-Principal Investigator: Kratochwill, Thomas
Questions about this project?
To answer additional questions about this project or provide feedback, please contact the program officer.
