Project Activities
The researchers designed Connecting Text by Inference and Technology (Connect IT), a technology-based inference-making intervention for students with reading disabilities, as well as a face-to-face version of the same inference-making intervention. They iteratively developed 26 lessons consisting of 5 inference-making and practice modules and tested individual lessons from the intervention to understand usability and feasibility. Next, they conducted small-scale experimental studies on the entire suite of lessons in the face-to-face version and for each module in the computer-delivered version to inform the final intervention design for pilot testing. Finally, they conducted a randomized controlled trial to test the computer-delivered intervention compared to a face-to-face version and business as usual.
Structured Abstract
Setting
The study was conducted in middle schools in Texas.
Sample
The development of the first iteration of the intervention included the participation of 5 teachers and 26 of their students. The following year, 166 students participated. For the tutor-led version of the intervention the entire suite of lessons was delivered by one teacher to 30 of her students in small groups. For the computer-delivered lessons, 136 students with or at risk for reading disability based on their English Language Arts (ELA) state literacy assessment and low performance on a standardized reading comprehension test were randomized to receive one of the 5 modules. In the final year, 165 students in sixth, seventh and eighth grades who had been identified by their schools as in need of reading intervention were screened and 155 of these students meeting additional screening criteria participated in the randomized controlled trial.
The intervention, Connect-IT, consists of 26 lessons focused on making inferences. The content includes text-connecting inferences (connecting pronouns with their referents, connecting ideas and sentences within the text and using text to infer the meanings of new words) and knowledge-based inference making, instructional routines, the use of avatars in need of students' help, key word and graphic organizers, and inference-eliciting questions. The instructional approach includes frequent retrieval through testing, interleaved practice (i.e., mixed practice on several related skills together instead of blocking on the same skill), cumulative practice, and immediate feedback.
Research design and methods
In the first year, the researchers iteratively developed lesson modules to be implemented by teachers with groups of two students and computer-based modules to understand the feasibility and fidelity of implementation of the intervention. Information collected from questionnaires and focus groups of teachers and students informed revisions to the lessons. In the next year, small-scale experimental studies were conducted to compare pre- and post-test inference-making accuracy and reading comprehension performance of students using each version of the intervention compared to a control group to prepare a final version of the intervention for the pilot study. These small-scale experiments also tested whether the teacher-led and computer-led versions of the intervention could be conducted in 30 minutes examined the optimal methods of presenting the information when converting the on-computer format to the tutor-led off-computer format. The pilot study used a randomized design to compare outcomes for students among the three groups—computer delivered Connect-IT intervention, tutor-led face-to-face version of Connect-IT, and business as usual.
Control condition
Students in the control condition received their school’s business-as-usual English Language Arts intervention instruction while participating in the pilot study.
Key measures
Revisions to the intervention were informed by data collected from teacher and student usability surveys, student and teacher focus groups, and fidelity assessments. The fidelity assessment captured important components of Connect-IT to document whether teachers used the intervention as intended. Participant screening measures included the Sight Word Reading Efficiency subtest of the Test of Word Reading Efficiency and the Reading Comprehension subtest of the Wechsler Individual Achievement Test (WIAT-3), also given at post-test. Pre- and post-test measures of inference making included the Bridging Inferences Task and a measure developed by the research team that assessed all four types of inferences in the intervention using new passages and questions.
Data analytic strategy
Descriptive statistics were used to summarize the feedback from teachers and students from the usability surveys. Data from the focus groups were summarized to provide feedback to the developers to inform lesson revisions. Student outcome data were analyzed using a multilevel generalized linear model. In addition to the main effect, analyses were also conducted to understand how learner characteristics (word reading ability, pretest reading comprehension, sex, English Learner status, anxiety, mind wandering, mindset for learning) moderate learner outcomes.
Key outcomes
The main findings of this study, as reported by the principal investigator, are as follows:
- The Connect-IT small group tutor-led version of the intervention was more effective for increasing inferential reading comprehension than was the computer version of Connect-IT or the typical teacher-led intervention provided by school personnel. However, intervention effects also varied depending on a number of student characteristics, which adds information regarding which interventions might be most effective for which students.
- The small group tutor-led version of Connect-IT was most effective for students with low levels of comprehension but adequate word reading (even if below grade level).
- Both the schools’ typical literacy interventions and the tutor-delivered Connect-IT intervention had larger positive effects on several comprehension measures for boys compared to the computer version of Connect-IT. These findings suggest that teacher supervision and interaction with teachers and peers might be particularly important for boys who are struggling with reading comprehension.
- Based on additional measures that were administered to a subset of the participants, the computer version of Connect-IT was more effective at improving reading comprehension compared to the school delivered intervention for students who self-identified as being more anxious. These effects may be related to how computer-delivered interventions remove perceived teacher and peer evaluation from the learning context.
- Students who reported themselves as being prone to high levels of mind wandering during reading and other academic tasks also benefitted from the computer-delivered Connect-IT intervention compared to the typical school-delivered intervention classes. These effects may be related to how the self-pacing of the computer version with many opportunities to respond and receive feedback on performance may benefit students who have difficulty with controlling their focus of attention during reading, particularly when other students are present.
People and institutions involved
IES program contact(s)
Project contributors
Products and publications
Publications:
ERIC Citations: Find available citations in ERIC for this award here.
Additional project information
Additional online resources: https://www.youtube.com/watch?v=rME6FiM49X0
Previous award details:
Related projects
Questions about this project?
To answer additional questions about this project or provide feedback, please contact the program officer.
