Project Activities
The researchers will iteratively develop and refine AT-ARC, conduct usability and feasibility tests, and run a pilot study of the fully developed intervention. They will also analyze the cost of the intervention.
Structured Abstract
Setting
Research will be conducted in Tennessee, Georgia, and other locations throughout the U.S.
Sample
Approximately 120 adult students in adult education programs with reading levels between 3rd to 8th grade equivalents will participate in the development phase, and approximately 375 (assuming 25 percent attrition) will participate in the pilot study, with continued enrollment until at least 300 adults complete 50 hours of exposure to AT-ARC lessons.
The previously developed AT-ARC is an internet-based program that uses three avatars, one of which is the adult learner, to introduce 35 reading comprehension strategy lessons (instruction and practice). Originally, teachers used AT-ARC in conjunction with a teacher-led classroom curriculum. During this project, the researchers will refine AT-ARC so that it can be a stand-alone intervention that instructors can use across a diverse range of adult literacy programs in the U.S. The researchers will also develop new components including additional reading comprehension lessons, digital literacy lessons, and professional development modules. The digital literacy lessons will address basic skills gaps in students' abilities to use technology, such as how to use a mouse and keyboard. The professional development modules will address issues such as integrating in-class use of AutoTutor-ARC software program with ongoing reading comprehension strategy instruction, aiding students in transitioning to remote (out of classroom) use of AutoTutor-ARC software by accessing the internet in public or private spaces, and helping teachers interpret dashboard results of student progress.
Research design and methods
The researchers will use an iterative process of design studies to develop the enhanced AT-ARC technology features and professional development modules. In phase 1, the researchers will focus on developing and refining digital skills modules, new practice texts and activities, a toolkit for instructors, and instructor professional development modules. In phase 2, they will implement a cycle of design studies to evaluate ease of use, utility, and feasibility of refinements to the refinements AT-ARC system, as well as the feasibility, satisfaction, and effectiveness of the professional development modules. During these phases, expert panels, instructors, and adult learners will provide feedback to help inform the revisions. After the researchers have the fully developed refined AT-ARC, the researchers will evaluate its promise and analyze the cost of AT-ARC's implementation. The pilot study will use a repeated-measures approach in which students are evaluated at different time points (such as after having completed 15 hours of instruction and after 30 hours of instruction).
Control condition
Given the repeated-measures research design, students will serve as their own controls.
Key measures
During the design phase, the researchers will collect indicators of student's engagement and learning including motivational surveys, log process data of time spent and performance. During the pilot study, they will measure literacy levels and potential gains using an existing comprehension measure, the SARA battery.
Data analytic strategy
The researchers will use an unconditional growth model that examines growth in SARA scores after the student completes 25 and 50 hours of exposure to the AT-ARC system and a growth model that includes intensity, time, and completion as main and interaction effects. In these models, the intercepts and slopes for time are modeled as mixed effects.
Cost analysis strategy
Researchers will conduct an ingredients-approach cost analysis on the fully developed intervention during the pilot study.
People and institutions involved
IES program contact(s)
Project contributors
Products and publications
Products: This project will develop AT-ARC and supporting materials, information about the cost of implementing the intervention, and peer-reviewed publications, presentations, and additional dissemination products (e.g., research briefs) that reach education stakeholders such as practitioners and policymakers.
Publications:
ERIC Citations: Find available citations in ERIC for this award here.
Select Publications:
Journal articles
Chen, S., Fang, Y., Shi, G., Sabatini, J., Greenberg, D., Frijters, J., & Graesser, A. C. (2021). Automated disengagement tracking within an intelligent tutoring system. Frontiers in Artificial Intelligence, 3, 1-16.
Fang, Y., Lippert, A., Cai, Z., Chen, S., Frijters, J., Greenberg, D., & Graesser, A. (2021). Patterns of adults with low literacy skills interacting with an intelligent tutoring system. International Journal of Artificial Intelligence in Education, 32, 1-26.
Graesser, A. C., Greenberg, D., Frijters, J. C., & Talwar, A. (2021). Using AutoTutor to track performance and engagement in a reading comprehension intervention for adult literacy students. Revista Signos. Estudios de Lingüística, 54(107).
Hollander, J., Sabatini, J., & Graesser, A. (2022). An intelligent tutoring system for improving adult literacy skills in digital environments. COABE Journal: The Resource for Adult Education, 10(2),Article 6, 59-64.
Hollander, J., Sabatini, J., Graesser, A., Greenberg, D., O'Reilly, T., & Frijters, J. (2023). Importance of learner characteristics in intelligent tutoring for adult literacy. Discourse Processes, 1-13.
Smith, E. H., Hollander, J., Graesser, A. C., Sabatini, J., & Hu, X. (2021). Integrating SARA Assessment with Reading Comprehension Training in AutoTutor. English Teaching, 76(1), 17-29.
Proceedings
Shi, G., Wang, L., Zhang, L., Shubeck, K., Peng, S., Hu, X., & Graesser, A. C. (2021, July). The Adaptive Features of an Intelligent Tutoring System for Adult Literacy. In International Conference on Human-Computer Interaction (pp. 592-603). Cham: Springer International Publishing.
Additional project information
Related Network Teams: In FY22, this project joined as a research team in the CREATE Adult Skills Network, which conducts research and dissemination to support the millions of U.S. adults who have low foundational skills.
Kick-off Webinar available for viewing here
Related projects
Questions about this project?
To answer additional questions about this project or provide feedback, please contact the program officer.
