Co-Principal Investigator: McMaster, Kristen

Purpose: This project developed the Technology-Based Early Language Comprehension Intervention (TELCI) for first- and second-grade students with or at risk for disabilities who experience reading comprehension difficulties. According to the 2013 National Assessment of Educational Progress, approximately a third of fourth-grade students struggle in the area of reading comprehension, making it crucial to address these difficulties in earlier grades. A disproportionate number of students with reading comprehension difficulties are from minority and low socio-economic backgrounds. Thus, there is a continued need for efforts to raise the reading achievement and prevent long-term negative reading outcomes among diverse learners who experience reading difficulties in high-needs schools (such as urban schools with high percentages of English learners and students receiving free and reduced lunch or a high percentage of students performing below reading proficiency level). In response to this need, the project developed the TELCI to improve reading comprehension of students in high-needs schools who experience comprehension difficulties in the early elementary years by developing inference-making skills.

Project Activities: The research team employed an iterative process to develop and refine TELCI. They developed the app and modules and conducted a series of field tests to identify specific components that optimize its usability, feasibility, and promise. In the final year, a small randomized controlled trial was used to test the intervention's feasibility for implementation in schools and promise for improving inference skills and subsequent reading comprehension for students in first and second grade who struggle with reading comprehension.

Key Outcomes: The main findings of this project, as reported by the principal investigator, are as follows:

• Students who completed TELCI showed improvement in inference making over the intervention period and benefited from the scaffolding and feedback that were provided during the intervention, though the improvement in inference-making skill was comparable to that of business-as-usual control students.
• The research team identified a subset of students for whom the intervention worked (responders) and found that responders outperformed the control students on a researcher-developed assessment of inference making. However, these results did not generalize to a standard language comprehension measure.
• There were no differences between groups on the distal reading comprehension measure.

Structured Abstract

Setting: The research took place in elementary schools in an urban and ethnically diverse district in Minnesota.

Sample: Approximately 285 first through second graders with difficulties in reading comprehension and 34 teachers participated in this research across the 3 years of the project. In addition, two members of school personnel with expertise in early literacy intervention and six parents were recruited to provide input and feedback on the development, usability, and feasibility of TELCI.

Intervention: TELCI is an interactive, cloud-based, software app aimed at improving reading comprehension by developing inference-making skills in young readers with difficulties in reading comprehension. The app has 24 learning modules that take 20 minutes each to complete. Each module engages students to (a) view age-appropriate videos (half fiction and half nonfiction), (b) learn key vocabulary words that are central to the main ideas in the video, (c) respond to inferential questions during (online) and after (offline) videos, and (d) receive scaffolding and specific feedback for each question. TELCI also includes eight transfer modules (20 minutes each) during which an interventionist reads a text (half narrative and half informational) aloud in a small group and students participate in an interactive questioning activity. Each week for 8 weeks, students complete four TELCI sessions, including three learning modules on the app and one transfer module.

Research Design and Methods: In year 1, the research team worked with teachers, school personnel (early literacy intervention experts), and parents to develop the TECLIapp and learning modules. A field test was conducted to examine initial usability and feasibility. Students were randomly assigned to receive the intervention with either online questioning or offline questioning and either fiction or nonfiction videos. Usability and feasibility data were collected and analyzed to inform revisions. In year 2, the research team developed and field tested TELCI transfer modules and conducted an experimental field trial to compare the feasibility and promise of the two questioning versions (online vs. offline questioning) of TELCI, with fiction and nonfiction videos. Sixty students were randomly assigned to either the online or offline questioning version of TELCI. Data from fidelity observations, student pre- and post-test measures, and usability assessments were examined to determine the most feasible and promising version of TELCI to be tested the following year. In year 3, the research team conducted a small, randomized controlled trial for the pilot study with 129 students assigned to either the intervention or control group. The researchers analyzed data to determine the promise of the intervention for improving proximal and distal student outcomes and the moderating effects of decoding and vocabulary.

Control Condition: In the pilot study, students in the control condition participated in business-as-usual language comprehension instruction.

Key Measures: To determine eligibility, the Clinical Evaluation of Language Fundamentals – 5th Edition (CELF-5) and the Gates-MacGinitie Reading Test (GMRT) were used to assess inference and reading comprehension skills, respectively. The research team developed several measures to assess usability and feasibility, including focus group protocols to gain initial input from school personnel and parents, questionnaires for teachers and school early literacy experts, intervention logs and bug sheets to note errors in intervention materials, and student surveys. Fidelity was assessed using a researcher-developed observational tool. Student inference skills, a proximal outcome, was assessed using a video comprehension measure developed by the research team in a previous project. Distal student outcomes included language and reading comprehension with standardized assessments. The CELF-5 was used to assess inference skills in non-reading contexts and the GMRT was used to measure reading comprehension. Possible moderators were assessed using the FAST-earlyReading battery to measure decoding, the CELF-5 to measure vocabulary, and the Minnesota Executive Function Scale to measure executive functions.

Data Analytic Strategy: Descriptive analyses were conducted on quantitative data to inform development and feasibility. To analyze qualitative data, researchers conducted a content analysis to determine patterns and themes across data sources. Two researchers analyzed the data independently and came to consensus regarding the salience of themes. Analysis of variance was used to compare different versions of the intervention in year 2 and to determine the promise of the intervention in year 3. Regression analysis was used to examine relations between intervention, proximal outcomes, moderators, and distal student outcomes in year 3.

Related IES Projects: Early Language Comprehension Individualized Instruction (R305A170242); Developing a Technology-Based Early Language Comprehension Intervention (R324A160064)

Products and Publications

ERIC Citations: Find available citations in ERIC for this award here.

Project Website: https://innovation.umn.edu/tutoring-system/

Select Publications

Journal articles

Kendeou, P., McMaster, K., Butterfuss, R., Kim, J., Bresina, B., & Wagner, K. (2020). The Inferential Language Comprehension (iLC) Framework. Topics in Cognitive Science, 12,256–273.

McMaster, K., Kendeou, P., Bresina, B. C., Slater, S., Wagner, K., White, M. J., ... & Umana, C. (2019). Developing an interactive software application to support young children's inference-making. L1-Educational Studies in Language and Literature, 1–30.