Co-Principal Investigators: Meyer, Bonnie J.F.; Lei, Pui-Wa; Johnson, Lori; Walpole, Sharon

Purpose: The purpose of this replication project is to test the efficacy of the Intelligent Tutoring system for the Structure Strategy (ITSS), an online tutor for fourth and fifth grade students that teaches them about different structures for text (e.g. comparison, cause and effect) to improve reading comprehension outcomes. One problem that many students who struggle to read may encounter is the inability to understand and remember what they read. The structure strategy is a method of understanding and remembering text by learning to identify different ways text can be organized, which in turn will help students to focus on the important ideas and arguments within an informational text. ITSS was developed during a previous IES-funded project (Intelligent Tutoring Using The Structure Strategy To Improve Reading Comprehension Of Middle School Students) and tested for efficacy in a prior IES-funded efficacy study (Efficacy and Replication Research on the Intelligent Tutoring System for the Structure Strategy—Rural and Suburban Schools Grades 4, 5, 7, and 8). Findings from this efficacy study indicated that ITSS improved students' reading comprehension outcomes for fourth and fifth graders. This replication study increases the time spent per week interacting with ITSS from the previous efficacy study, and focuses specifically on students from low-income schools.

Project Activities: During the initial year of the study, the project team will finalize study measures and professional development. Over Years 2–3, more than 200 teachers and their students in 36 elementary schools will participate in this randomized field trial to test the efficacy of ITSS. In this study, researchers will randomly assign all students in a school to receive either ITSS or to receive the regular classroom practices without ITSS. Students in the treatment group will use ITSS twice a week for thirty minutes each time during the school day for one school year. Teachers will receive professional development before implementing the intervention as well as on-site coaching as necessary. Students in both groups will be assessed before the study, at the end of receiving their intervention, and in the fall of the school year following the end of the intervention.

Products: Researchers will produce evidence of the efficacy of ITSS intervention for fourth and fifth grade children attending high poverty schools. In addition, researchers will produce peer reviewed publications.

Project Website: http://itss.psu.edu/itss

Structured Abstract

Setting: This project will take place in 36 high poverty schools in Texas and Pennsylvania.

Sample: Approximately 216 teachers and their 3,888 fourth and fifth grade students will participate in the study.

Intervention: The individualized-adaptive ITSS that will be used in this project provides over 100 online interactive lessons on five text structures: comparison, problem and solution, cause and effect, sequence, and description. The lessons teach how to recognize the text structures by using key words and how to use knowledge of text structure to understand and remember the content of informational text. The adaptive intelligent tutor customizes scaffolding and informative feedback based on student performance on individual lessons. This individualized-adaptive version of ITSS customizes pathways through ITSS instruction by using online assessments and performance in lessons to select a standard lesson, remediation lesson, or enrichment lesson. ITSS includes the online intelligent tutor, teacher professional development, and classroom materials such as laminated key-shaped flashcards outlining the key points in structure strategy and classroom posters. The online ITSS includes an administrator toolbox that allows teachers to view student responses and to print reports on student progress.

Research Design and Methods: This project uses a multi-site cluster randomized trial. Nine districts will participate in the project and schools will be randomly assigned within district to the ITSS or the control condition. Year 1 of the project will involve finalizing measures and professional development. All fifth grade teachers will be invited to participate with their students in Year 2 of the study, and all fourth grade teachers will be invited to participate with their students in Year 3. Students in the ITSS group will use the intelligent tutor twice a week thirty minutes each time. Students will be assessed at the beginning of the school year before participating in the study, at the end of the school year after the intervention is complete, and in the fall of the following school year as a delayed posttest.

Control Condition: In the control condition, students receive standard classroom practices in place at the school.

Key Measures: Key student outcome measures used in the project will include: the Gray Silent Reading Test and experimenter-designed measures of reading comprehension. Additional student measures include survey questions about motivation to read. Implementation fidelity and comparison group practices will be assessed using observations and teacher logs. A researcher-designed measure of motivation to read, self-efficacy with computer usage, and self-efficacy for using signal words and writing main ideas will be used to assess affective outcomes.

Data Analytic Strategy: Multilevel modeling will be used to examine the efficacy of ITSS on student reading outcomes. Students will be Level 1, classrooms Level 2, schools Level 3, and districts Level 4. The random assignment occurs at Level 3. Pre-test scores will be included as covariates in models with the post-test used as the outcome of interest, and post-test scores will be included as covariates in models with the delayed post-test as the outcome of interest. Fidelity observation checklists, teacher logs, and teacher surveys will be coded and used to determine the degree to which teachers in both conditions engage in comprehension instruction. Measures of affective outcomes will be used as mediators in the association between ITSS and student's comprehension outcomes.

Related IES Projects: Efficacy Trial of Text Structure Strategy Instruction on the Web for Spanish-Speaking English Learners (SWELL) in Grades 4 and 5 (R305A180060); Improving Reading Comprehension of Middle Grades English Language Learners by Combining Structure Strategy with Web-Based Adaptive Tutoring for EL Learners (SWELL) (R305A130704); Efficacy and Replication Research on the Intelligent Tutoring System for the Structure Strategy–Rural and Suburban Schools Grades 4, 5, 7, and 8 (R305A080133); Intelligent Tutoring Using The Structure Strategy To Improve Reading Comprehension of Middle School Students (R305G030072); Promoting Reading Comprehension and Learning With Multimodal Science Texts (PRISM) (R305A220015)

Products and Publications

Hudson, A.K., Owens, J.K., Moore, K.A., Lambright, K. & Wijekumar, K. (2021). "What is the main idea?": Using text structure as a framework for accelerating strategic comprehension of text. Reading Teacher, 75(1), 113. doi.org/10.1002/trtr.2016

Meyer, B. J., Wijekumar, K., & Lei, P. (2018). Comparative signaling generated for expository texts by 4th–8th graders: Variations by text structure strategy instruction, comprehension skill, and signal word. Reading and Writing, 31(9), 1937–1968.

Wijekumar, K., Beerwinkle, A., McKeown, D., Zhang, S., & Joshi, R. M. (2020). The "gist" of the reading comprehension problem in grades 4 and 5. Dyslexia: An International Journal of Research and Practice | ISSN: 1076–9242 PMID: NLM31903668.