|Project SCORE IT: Developing and Evaluating Interactive Technology to Support Self-Monitoring and Data-Based Decision Making in the Classroom
|University of Iowa
|Technology for Special Education [Program Details]
|3 years (7/1/2016-6/30/2019)
|Development and Innovation
Co-Principal Investigators: Joseph Wehby (Vanderbilt University)
Purpose: The purpose of this project is to further develop SCORE IT, a technology-based self-monitoring intervention for use in middle school classrooms, by adding critical data-based decision-making support for teachers working with students who have challenging behavior. Although tools exist for teachers to evaluate a student's response to academic interventions, tools are far less common in the area of behavior. The enhanced SCORE IT is intended to increase academic engagement and decrease problem behavior by providing teachers an efficient data collection mechanism with explicit rules to determine a student's responsiveness to behavioral intervention and make accurate further behavioral programming decisions.
Project Activities: A series of iterative activities will be completed over a 3-year period to develop the components of SCORE IT and examine the feasibility of the intervention in middle schools. These activities include soliciting ongoing input and feedback from an advisory board and school focus groups. In Year 1, the team will develop SCORE IT decision rules and develop the prototype. In Year 2, the team will test the prototype with students and teachers who will use SCORE IT on iPads within a school setting. In Year 3, the team will conduct a randomized controlled trial to evaluate the impact of SCORE IT on students' engagement, behaviors, and academic performance.
Products: The products of this project include a fully developed intervention (SCORE IT), peer-reviewed publications, and presentations.
Setting: This study will take place in middle schools in Iowa and Tennessee.
Sample: Ten general educators, special educators, and behavior specialists from Grades 5–8 will participate in the focus groups. Participants for the prototype testing include 20 student-teacher pairs from Grades 5–8. A new group of approximately 72 student-teacher pairs (36 at each site) will participate in the pilot study. Students will be eligible if they have no significant cognitive impairment and have behavioral goals included in their individualized education programs (IEPs).
Intervention: SCORE IT is an existing self-monitoring app that (a) enables students to evaluate and record their behavior in order to improve their self-regulation skills and (b) allows teachers to evaluate students on the same self-monitored behaviors. SCORE IT has three components: (1) Student component: Self-monitoring allows up to three student behaviors to be monitored. The student and teacher can work together to identify relevant behaviors to monitor if preprogrammed default behaviors are not applicable. (2) Teacher component: Parallel procedures plus feedback allows teachers to complete the same recording procedures as the student component and then converse with the student about the rating and the rating agreement between the teacher and student. (3) Tracking progress allows the teacher and student to use an automated bar graph with a goal line to evaluate progress. During this project, the team will build data analysis and decision-making tools into the SCORE IT app to facilitate a teacher's determination of a student's responsiveness to an intervention and subsequent next steps (e.g., changing the goal, changing the frequency of self-monitoring, discontinuing the intervention, referring the student for further evaluation).
Research Design and Methods: In Year 1, the team will develop SCORE IT decision rules and develop the prototype by first conducting a review of the existing literature on self-monitoring, behavioral goal-setting, and Check-In/Check-Out (an existing behavior intervention) procedures. The team will also incorporate feedback from an expert advisory panel and focus groups of educators (e.g., general and special educators, and behavior specialists) in Grades 5–8. In Year 2, the team will test the prototype in school settings with students and teachers who will use SCORE IT on iPads over the course of several days. Researchers will conduct observations, examine field notes, and conduct semi-structured interviews to evaluate the usability and feasibility of SCORE IT. In Year 3, the researchers will conduct a cluster randomized controlled trial, stratified by grade level, in which classes/teachers serve as the unit of randomization (approximately 72 classrooms) to investigate the promise of SCORE IT at improving student outcomes.
Control Condition: In the comparison condition, teachers and their students will receive business-as-usual instruction, services, and professional development.
Key Measures: Student behavior outcomes will be observed using the Multiple Option Observation System for Experimental Studies (MOOSES) and student academic outcomes will be measured with curriculum-based reading measures. In addition, school records related to disciplinary referrals will be collected. Fidelity will be conducted via fidelity checklists, and teacher and student acceptability with SCORE-IT will be assessed via surveys and interviews.
Data Analytic Strategy: Qualitative data analysis techniques will be used for focus group and expert feedback. Descriptive analyses of fidelity will also be conducted. Analysis of variance will be used to assess the promise of the intervention's impact in the pilot study. Paired samples t-tests will determine if teacher and student acceptability ratings change after implementation of SCORE IT.