Co-Principal Investigator: Clemens, Nathan; Vaughn, Sharon; Roberts, Greg

Purpose: A large direct contributor to comprehension problems in middle school students with or at risk for reading disabilities is pervasive difficulty in making inferences necessary for understanding what is read. Because literacy attainments in middle school are highly predictive of postsecondary success, middle school provides a later developmental window within which to intervene for struggling adolescent readers. Consequently, a technology-based inference-making intervention, Connect-IT-Computer, and an interventionist-led version, Connect-IT-Teacher, were developed for middle school students with or at risk for reading disabilities in an IES Development and Innovation project (Barnes, PI). Promising findings from an underpowered RCT provide the rationale for the current proposal which has the following aims: (1) to test the efficacy of Connect-IT-Computer and Connect-IT-Teacher for middle school students with or at risk for reading disability, (2) to compare the efficacy of the two versions of Connect-IT, (3) to test mediators and moderators of hypothesized impacts of the interventions; and (4) to compare the cost and effectiveness of Connect-IT-Teacher and Connect-IT Computer versus what schools would provide in the absence of these programs.

Project Activities: In Year 1, the research team will recruit schools and observe business as usual (BAU) classrooms to gain an understanding of the BAU practices in the schools in which the efficacy study will be conducted. In Year 2, researchers will conduct screening, pretesting, intervention implementation, and fidelity observations in the fall. They will collect follow-up testing in February–March of the Spring semester. In Year 3, most of the time will be devoted to continued data processing and entry, including transcriptions of the Connect-IT-Teacher audio recordings, coding transcriptions using the fidelity form, and post-processing of individual student Connect-IT-Computer data from within the lessons. In Year 4 researchers will finalize data coding and analysis and complete the cost effectiveness analysis, and dissemination of findings.

Products: Products include information about the efficacy of the Connect-IT-Computer and Connect-IT-Teacher intervention over business as usual intervention as well as in comparison to each other, and related costs and cost-effectiveness. The project will also result in a released final data set, peer-reviewed publications and presentations as well as additional dissemination products that reach education stakeholders such as practitioners and policymakers.

Structured Abstract

Setting: The research will take place in schools serving students in grades 6–8 in Tennessee and Texas.

Sample: The participants include 360 students in grades 6-8 who did not pass their state English Language Arts (ELA) test and who meet word reading and reading comprehension screening.

Intervention: The Connect-IT intervention consists of 26 lessons with one introductory lesson and subsequent inference instruction that is provided in five modules with five lessons each as follows: Pronoun Inferences; Text-Connecting Inference; Meaning-from-Context Inferences; Knowledge-Based Inferences, and Mixed Practice. Instructional routines and content are based on research on inferential processing as well as cognitive learning principles such as retrieval practice including interleaved practice and feedback. Connect-IT-Computer delivers the lessons on computer supervised by project interventionists. Connect-IT-Teacher delivers the same content and instructional routines in a small group format led by project interventionists.

Research Design and Methods: The researchers will conduct a multi-site randomized controlled trial, where students are blocked and randomized within school, to determine the efficacy of Connect-IT-Computer and Connect-IT-Teacher for middle school students with or at risk for reading disability. The design is partially nested with all students nested in school and treatment students nested in school and in intervention cluster.

Control Condition: The control condition will be business as usual school-provided intervention classes.

Key Measures: Researchers will use several measures of reading comprehension (the Gates-MacGinitie Reading Test – Reading Comprehension subtest, Wechsler Individual Achievement Test III – Reading Comprehension Subtest and Woodcock Johnson Reading Mastery Test III –Passage Comprehension Subtest). The proximal measure (Connect-IT Inferential Reading Comprehension Assessment or CIRCA) will be used to assess all inference types instructed in Connect-IT. Student word reading skills will be measured with the Sight Word Efficiency and Phonemic Decoding Efficiency subtests of the Test of Word Reading Efficiency (TOWRE-2). Researchers will also measure text connecting inferences (Bridging Inferences Task-Bridge-IT), vocabulary inferences (Comprehensive Assessment of Spoken Language 2nd edition-Meaning from Context) and knowledge-based inferences (CASL-2 Inference). Moderator variables include measures of internal control of attention during reading (Mind-wandering Questionnaire, Probe-caught mind wandering task), anxiety (The Multidimensional Anxiety Scale for Children, 2nd Edition), word reading efficiency (TOWRE-2), and vocabulary and world knowledge (Kaufman Brief Intelligence Test-2 Verbal Knowledge subtest). Data will also be collected on the cost of the intervention implementation and BAU interventions.

Data Analytic Strategy: The main and conditional (mediated and moderated) effects of treatment(s) will be tested using multilevel regression and structural equation modeling. A cost-effectiveness analysis of Connect-IT Teacher and Connect-IT Computer will be conducted using the ingredients method to determine the total cost and the cost per 1 SD gain in key outcomes for each version of the program over BAU intervention.

Related IES Projects: Project Connect-IT (Connecting Text by Inference and Technology): Development of a Text-Integration Intervention for Middle School Students with Comprehension Difficulties (R324A170150); Understanding Malleable Cognitive Processes and Integrated Comprehension Interventions for Grades 7–12 (R305F100013)