|Title:||Developing and Implementing a Technology-Based Reading Comprehension Instruction System for Adult Literacy Students|
|Principal Investigator:||Sabatini, John||Awardee:||University of Memphis|
|Program:||Postsecondary and Adult Education [Program Details]|
|Award Period:||4 years (09/01/2020 – 08/31/2024)||Award Amount:||$1,399,882|
|Type:||Development and Innovation||Award Number:||R305A200413|
Co-Principal Investigators: O'Reilly, Tenaha; Greenberg, Daphne; Graesser, Arthur
Related Network Teams: In FY22, this project joined as a research team in the CREATE Adult Skills Network, which conducts research and dissemination to support the millions of U.S. adults who have low foundational skills and includes the following other projects: Adult Skills Network Lead(R305N210014), Teaching Skills That Matter (TSTM)–SkillBlox Research Team(R305N210025), Adult Numeracy in the Digital Era: Adaptive Technology for Quantitative and Digital Literacy (R305N210029), Writing in Adult Secondary Education Classes (W-ASE) (R305N210030), Adult Skills Assessment Program: Actionable Assessments for Adult Learners (R305N210031), Content-Integrated Language Instruction for Adults with Technology Support(R305N210032)
Purpose: In this project, researchers will refine and pilot test an interactive, online reading comprehension program for adult literacy students reading between the 3rd- and 8th-grade level. This program, called AutoTutor for Adult Reading Comprehension (AT-ARC), was previously part of a hybrid lesson plan developed through the Center for Struggling Adult Learners (R305C120001) but will be refined to be a stand-alone comprehension practice tool. Roughly one in six adults in the United States have literacy skills at a low level of proficiency and may seek instruction through adult education programs. However, these programs are often under-resourced, may lack appropriate instructional materials, and may have instructors with few professional development resources. AT-ARC will not only provide appropriate materials for adult learners to practice reading comprehension strategies but also address learners' digital literacy needs and instructors' needs for professional development.
Activities: The researchers will iteratively develop and refine AT-ARC, conduct usability and feasibility tests, and run a pilot study of the fully developed intervention. They will also analyze the cost of the intervention.
Products: This project will develop AT-ARC and supporting materials, information about the cost of implementing the intervention, and peer-reviewed publications, presentations, and additional dissemination products (e.g., research briefs) that reach education stakeholders such as practitioners and policymakers.
Setting: Research will be conducted in Tennessee, Georgia, and other locations throughout the US.
Sample: Approximately 120 adult students in adult education programs with reading levels between 3rd to 8th grade equivalents will participate in the development phase, and approximately 375 (assuming 25 percent attrition) will participate in the pilot study, with continued enrollment until at least 300 adults complete 50 hours of exposure to AT-ARC lessons.
Intervention: The previously developed AT-ARC is an internet-based program that uses three avatars, one of which is the adult learner, to introduce 35 reading comprehension strategy lessons (instruction and practice). Originally, teachers used AT-ARC in conjunction with a teacher-led classroom curriculum. During this project, the researchers will refine AT-ARC so that it can be a stand-alone intervention that instructors can use across a diverse range of adult literacy programs in the U.S. The researchers will also develop new components including additional reading comprehension lessons, digital literacy lessons, and professional development modules. The digital literacy lessons will address basic skills gaps in students' abilities to use technology, such as how to use a mouse and keyboard. The professional development modules will address issues such as integrating in-class use of AutoTutor-ARC software program with ongoing reading comprehension strategy instruction, aiding students in transitioning to remote (out of classroom) use of AutoTutor-ARC software by accessing the internet in public or private spaces, and helping teachers interpret dashboard results of student progress.
Research Design and Methods: The researchers will use an iterative process of design studies to develop the enhanced AT-ARC technology features and professional development modules. In Phase 1, the researchers will focus on developing and refining digital skills modules, new practice texts and activities, a toolkit for instructors, and instructor professional development modules. In Phase 2, they will implement a cycle of design studies to evaluate ease of use, utility, and feasibility of refinements to the refinements AT-ARC system, as well as the feasibility, satisfaction, and effectiveness of the professional development modules. During these phases, expert panels, instructors, and adult learners will provide feedback to help inform the revisions. After the researchers have the fully developed refined AT-ARC, the researchers will evaluate its promise and analyze the cost of AT-ARC's implementation. The pilot study will use a repeated-measures approach in which students are evaluated at different time points (such as after having completed 15 hours of instruction and after 30 hours of instruction).
Cost Analysis: Researchers will conduct an ingredients-approach cost analysis on the fully developed intervention during the pilot study.
Control Condition: Given the repeated-measures research design, students will serve as their own controls.
Key Measures: During the design phase, they will collect indicators of student's engagement and learning including motivational surveys, log process data of time spent and performance. During the pilot study, researchers will measure literacy levels and potential gains using an existing comprehension measure, the SARA battery.
Data Analytic Strategy: The researchers will use an unconditional growth model that examines growth in SARA scores after the student completes 25 and 50 hours of exposure to the AT-ARC system and a growth model that includes intensity, time, and completion as main and interaction effects. In these models, the intercepts and slopes for time are modeled as mixed effects.
Project Website: https://adulted.autotutor.org/
Chen, S., Fang, Y., Shi, G., Sabatini, J., Greenberg, D., Frijters, J., & Graesser, A. C. (2021). Automated disengagement tracking within an intelligent tutoring system. Frontiers in Artificial Intelligence, 3, 1–16.
Fang, Y., Lippert, A., Cai, Z., Chen, S., Frijters, J., Greenberg, D., & Graesser, A. (2021 ). Patterns of adults with low literacy skills interacting with an intelligent tutoring system. International Journal of Artificial Intelligence in Education, 32, 1–26.
Smith, E. H., Hollander, J., Graesser, A. C., Sabatini, J., & Hu, X. (2021). Integrating SARA Assessment with Reading Comprehension Training in AutoTutor. English Teaching, 76(1), 17–29.