Project Activities
First, the project team will iteratively develop capabilities within an existing innovative mobile technology (INFLO) and will integrate an LLM into the system. As part of development, the project team will conduct human-centered design research to increase accessibility and usability, improve product functionality, and inform initial testing studies. Next, the project team will use data engineering, machine learning, and natural language processing to explore how active learning interacts with individual differences in postsecondary settings. The researchers will also use rapid-cycle experimentation to explore impacts of student behaviors, technology features, and generative learning activities on learning outcomes facilitated through the use of Terracotta and the Arizona State University Learning at Scale infrastructure.
Structured Abstract
Setting
Research will take place at two large, public universities, Arizona State University and Indiana University.
Sample
The project team aims to include over 100,000 students in the proposed research.
Active L@S is a new technological capability built on top of an existing mobile technology (INFLO) that will provide adaptive and generative learning activities leveraging LLMs. Active L@S will be assimilated into Canvas LMS and can be deployed to students on their mobile devices. It will personalize the learning experience using generative learning prompts that encourage students to retrieve knowledge at spaced intervals and will provide real-time, adaptive feedback across multiple domains.
Research design and methods
First, the project team will build the technical capability to implement active learning, automated scoring, and feedback methods, and will integrate the product into Canvas LMS. Next, the project team will develop the active learning prompts and AI algorithms. During development, the project team will adopt a learning engineering approach engaging various stakeholders like students, instructors, and instructional designers. Using a human-centered design process, the project will combine usability studies, focus studies, and other iterative processes. Finally, the project team will conduct multiple rounds of a/b testing to test and enhance the effects of Active L@S on learning outcomes.
Control condition
The control condition for all of the experimental studies will be a group that interacts with the prior version of the mobile technology (INFLO) that does not have the generative learning and adaptive feedback enhancements.
Key measures
The project team will collect student demographic data, course-level data, assignment-level data such as discussion board data and written assignments, single ease questionnaire, satisfaction questionnaire, subjective usability questionnaire, system usability scale, net promoter score, INFLO satisfaction scale, and INFLO adoption scale.
Data analytic strategy
The project team will use machine learning and natural language processing to explore strategies for making active learning accessible and impactful among diverse student populations. Bayes Factors analysis, and other data science techniques will be applied to evaluate the effectiveness and outcomes of the learning strategies and tools used with diverse learners to account for individual and contextual factors.
Cost analysis strategy
The research team will conduct a thorough cost-analysis throughout the project, considering development, maintenance, scalability, and integration of Active L@S. They will also conduct a comprehensive cost-benefit analysis to determine the return on investment for enhancing student outcomes. A sensitivity analysis will also be undertaken to factor in uncertainties.
People and institutions involved
IES program contact(s)
Partner institutions
Indiana University
Products and publications
Products: Active L@S will be assimilated into the Canvas learning management system (LMS). Research findings will be shared with academic and practitioner communities through peer-reviewed publications and conference presentations.
Additional project information
Partner Institution(s): Indiana University, INFLO, Clevent AI Technology LLC
Related projects
Questions about this project?
To answer additional questions about this project or provide feedback, please contact the program officer.
