Project Activities
In Year 1, researchers will develop and validate the REP Integrity Protocol, an implementation fidelity observation tool. This tool, which will measure each component of the intervention, will be developed and refined through focus groups with educators and content experts. Researchers will use the same procedures to refine the REP Intervention Protocol (which guides implementation of REP), soliciting perceptions from experts on the intervention components, intervention usability, feasibility, contextual fit, understandability, and promise for impacting student oucomes. In Year 2, the feasibility of REP procedures will be evaluated via small-scale trials, where end users “try out” REP and provide feedback via feasibility ratings and focus group interviews. In Year 3, a series of single-case design studies will be used to conduct component analyses of each of the three major REP components to determine its ability to generate positive student outcomes. Based on results of these trials, revisions will be made to enhance REP's fidelity, feasibility, and usability. In Year 4, a randomized controlled trial pilot study will be used to determine the promise of the refined REP for improving student skill use, internalizing behavior, and academic outcomes.
Structured Abstract
Setting
This research will take place in elementary and middle schools implementing PBIS in Wisconsin.
Sample
Participants in the initial REP Integrity Protocol review include five experts in the REP intervention components for content validation and 20 graduate students with relevant training who will participate in inter-observer reliability tests. For the development of the REP Intervention Protocol, 5 content experts, 10 school personnel (such as teachers and counselors), and 10 parents will participate. A total of 12 students with internalizing problems, their teachers, and their parents), as well as two student support personnel (such as psychologists and counselors), across four schools will participate in the feasibility studies. For the component analyses, 12 students with internalizing problems, their teachers, and their parents across three schools, as well as three student support personnel (one per school), will participate. The pilot study will include 120 students with internalizing problems, their teachers, and their parents across six schools as well as approximately 10 student support personnel.
Intervention
The REP is composed of three components. The first is the cognitive-behavioral instructional curriculum intended to promote student acquisition of social-emotional skills. Teaching follows a direct instruction model, incorporating verbal instruction, modeled instruction, and rehearsal. The second component corresponds with Check In/Check Out (CICO), an existing intervention that is intended to promote the maintenance and generalization of acquired skills. The third component is parent skills training, through which parents learn skills that promote CICO implementation and facilitate positive parent-child relations. School-based student support personnel deliver REP components; teachers provide performance feedback and positive attention as part of CICO.
Research design and methods
The first 3 years of the project will involve iterative intervention development. In Year 1, the research team will develop and refine the REP Integrity Protocol and the REP Intervention Protocol. Items on the REP Integrity Protocol will undergo content validation by experts in the REP intervention components and will then be evaluated for inter-observer reliability. Across two focus groups, experts and practitioners will review the REP Intervention Protocol and provide critical feedback regarding (a) the strengths and weaknesses of its structure and format, (b) potential revisions or additions that might enhance REP's effectiveness, and (c) REP feasibility for use within school settings. In Year 2, field testing will be used to obtain formative feedback. After researchers screen and identify students for internalizing problems, their teachers and school's student support personnel will be trained to implement relevant REP components. A series of A-B single-case design studies will be used to try out and examine the REP program components potential impact on student academic engagement and social behavioral outcomes. Focus groups with teacher, parents, and support personnel will be conducted following the completion of all feasibility trials. In Year 3, three multiple-baseline across subjects single-case design studies will be used for the component analysis. Each study will represent an “add-in” component analysis that examines the necessity of two REP components over and above one component implemented in isolation. In Year 4, the research team will conduct a small randomized controlled trial to pilot test the revised REP intervention. Students will be randomly assigned to the intervention group or to a control group. Those in the intervention group will receive REP for approximately 10 weeks. The research team will collect data at pre-, post-, and 1-month follow-up sessions to evaluate the promise of REP on proximal outcomes, including student social-emotional skills and internalizing problems, as well as distal outcomes, including social/academic engagement, social-emotional competence, and academic achievement. The research team will also conduct cost and cost-effectiveness analyses to help schools and districts understand the monetary costs of implementing REP, including the costs relative to observed student outcomes.
Control condition
For the pilot study, students in the control group will receive services as usual.
Key measures
For the initial iterative development process, data will be collected through focus groups and questionnaires. For the field testing, the research team will use a variety of measures to assess implementation fidelity and feasibility, including the REP Integrity Protocol, the BASC-3 Behavioral and Emotional Screening System to screen students, Usage Rating Profile—Intervention, Revised to measure intervention feasibility, and a semi-structured questionnaire to guide focus group interviews. Student outcome measures include the Internalizing Behavior Observation Protocol to measure internalizing behaviors as well as social and academic engagement, the Behavior Assessment System for Children, Third Edition as a supplemental measure of internalizing behavior, the Devereux Student Strengths Assessment to measure social-emotional competence, and the Perceived Control of Internal States Scale and self-report logs to measure students' self-reported use of social-emotional skills.
Data analytic strategy
Focus group and interview data will be analyzed qualitatively. Descriptive statistics will be used to evaluate feasibility measures. Single-case data will be evaluated using visual analysis, effect sizes using Tau-U statistics, and advanced statistical analyses, including the use of randomization within the design structure of SCD. Pilot study data will be examined via two-level hierarchical linear modeling. REP cost-effectiveness will be examined with the Incremental Cost-Effectiveness Ratio (ICER).
People and institutions involved
IES program contact(s)
Products and publications
Products: The products of this project will include a fully developed REP model for students with internalizing disorders and evidence of its promise for improving student cognitive-behavioral skill use, internalizing behavior, and academic outcomes. The project will result in peer-reviewed presentations and publications as well as additional dissemination products that reach education stakeholders such as practitioners and policymakers.
Supplemental information
Co-Principal Investigators: Lewis, Timothy; Eklund, Katie
Questions about this project?
To answer additional questions about this project or provide feedback, please contact the program officer.