Co-Principal Investigator: Basham, James; Choi, Jeong Hoon; Harsh, Robert

Purpose: The purpose of this project is to develop and pilot test a decision support system (DSS) that guides school teams in using data to implement an integrated multi-tiered system of support (MTSS) and improve outcomes for students with or at risk for a disability. Integrated MTSS is intended to provide both behavior and academic services; but despite the theoretical advantages of integrating these services, many schools struggle to implement MTSS at a high level of fidelity. This project will develop and test a technology tool, the DSS, to support schools in attaining high fidelity of MTSS implementation. The DSS will include a digital system (DataWall)that integrates data from multiple sources and an evidence-based problem-solving approach (Team-Initiated Problem Solving; TIPS) that will enable school teams to effectively utilize the digital system to identify problems and intervention solutions.

Project Activities: The research team will use an iterative process to develop and test the DSS. In Year 1 they will develop the DSS prototype and implementation plan in collaboration with their partner schools. In Year 2 they will conduct an initial field test of the DSS prototype and refine the system based on feedback to strengthen its usability, feasibility and acceptability. In Years 3 and 4 the research team will conduct a pilot randomized controlled trial to analyze the costs of implementing DSS and to test its promise for improving outcomes for students with or at risk for disabilities.

Products: The project will result in a fully developed DSS for school teams to use in making decisions about services provided to students with or at risk for disabilities. In addition, the project will result in peer-reviewed publications and presentations as well as additional dissemination products that reach education stakeholders such as practitioners and policymakers.

Structured Abstract

Setting: The research will take place in elementary schools in Kansas.

Sample: Across Years 2-4, approximately 13 elementary schools and school teams within each school will participate. School teams may include grade-level teams and other teams at the school that are involved in the implementation of MTSS, such as intervention teams, MTSS teams, and leadership teams. The sample for the pilot study in Years 3 and 4 will also include 240 students with disabilities (20 students per each of the 12 participating schools).

Intervention: The DSS intervention includes two components—DataWall and TIPS. DataWall is an existing integrated data system that enables schools to (a) link educational databases together; (b) concurrently chart various types of data; and (c) build summary reports of data at the district, school, grade, and classroom level. TIPS is an evidence-based approach that supports positive team functioning by providing a meeting structure (that includes defining roles, following fixed start/end times, using agendas and meeting minutes, etc.) and a protocol for team problem solving. The problem-solving protocol includes the following five steps: (a) identifying problems, (b) develop hypotheses, (c) selecting solutions, (d) implementing action plans, and (e) revising action plans based on results. In the current project, the research team will enhance the functionality of DataWall by adding features to allow the import and export of data into DataWall, to recognize patterns in behavior data with academic data, and to analyze levels of MTSS implementation using fidelity data. The DSS will allow school teams to triangulate data using DataWall to better match interventions to students' needs.

Research Design and Methods: This project will be conducted in three phases. In Phase 1, the DSS prototype will be developed with feedback from a research advisory group. Specifically, the new features of DataWall will be added and DataWall will be integrated into TIPS. In addition, the research team will develop the Multi-Tiered System of Support-Fidelity Checklist(MTSS -FC). This web-based self-rating checklist will be used to assess the following components of MTSS implementation: (a) school leadership around MTSS, (b) three-tiered intervention structure, (c) use of evidence-based interventions, (d) formal data-based decision rules for matching student needs to supports, (e) universal screening, (f) progress monitoring of students receiving additional (Tier 2) and intensified (Tier 3) levels of support, and (g) use of intervention fidelity measures. In Phase 2, researchers will conduct a field test of the DSS in one elementary school to examine its usability, feasibility, and acceptability among school team members using the DSS in order to refine the technology. Focus groups will be conducted at the end of the field test to allow school teams to provide feedback on the intervention's usability and feasibility. In Phase 3, a pilot study of DSS will be conducted using a cluster randomized controlled trial in 12 elementary schools (6 immediate implementers vs. 6 waitlist implementers) to evaluate its cost, usability, feasibility, acceptability, and promise for improving MTSS implementation and the math, reading, and behavior outcomes of students with or at risk for a disability.

Control Condition: Schools in the control condition will engage in typical implementation of MTSS.

Key Measures: TIPS-FC will be used to measure fidelity of TIPS implementation and the use of DataWall will be assessed using system data generated by the program. The Decision Observation, Recording and Analysis measure will be used to assess team problem solving. The MTSS -FC will be used to assess the quality of MTSS implementation. The Measures of Academic Progress will be used to assess students' reading and math performance. Administrative data will be used to measure behavior (office discipline referrals) and other student outcomes (such as attendance). Focus group data will be collected from school teams to assess their perspectives on the intervention's usability and feasibility.

Data Analytic Strategy: Focus group data will be analyzed using content analysis to identify themes and inform revisions to the intervention. Descriptive analyses will be used to summarize quantitative data related to the intervention's usability, feasibility and acceptability. Multilevel modeling will be used to analyze the effects of the intervention on MTSS fidelity of implementation, team problem solving, and student outcomes. The ingredients method will be used to determine the costs of implementing the intervention.