|Title:||Using Adaptive Practice to Improve Recall and Understanding in Postsecondary Anatomy and Physiology|
|Principal Investigator:||Pavlik Jr., Philip||Awardee:||University of Memphis|
|Program:||Postsecondary and Adult Education [Program Details]|
|Award Period:||3 Years (07/01/2019 - 06/30/2022)||Award Amount:||$1,240,151|
|Type:||Development and Innovation||Award Number:||R305A190448|
Co-Principal Investigators: Banker, Amanda M.; Olney, Andrew M.
Purpose: This project will further develop and refine an online system, Mobile Fact and Concept Training System (MoFaCTS), to help students in community college introductory anatomy and physiology courses better understand and remember course content. Introductory anatomy and physiology courses prevent many students from advancing in certificate or degree programs because they struggle to comprehend course texts or connect concepts to a broader understanding of course content. MoFaCTS will help increase students' comprehension and retention of material through sequenced formative practice exercises. These exercises help students learn vocabulary and form mental representations of text content through testing-based practice with supported learning.
Project Activities: The researchers will iteratively develop new components of the online system for both students and teachers. The system will leverage existing textbook material and instructor input to develop the interactive practice exercises. The development will also include usability and feasibility testing and a culminating pilot study.
Products: Products include a fully developed MoFaCTS for postsecondary courses, a cost analysis of MoFACTS, and dissemination materials, including peer-reviewed publications.
Setting: The classroom research will take place in a community college in Tennessee.
Sample: Approximately 1280 community college students and 4 instructors will participate throughout the project. Ten classrooms with approximately 20 students each will participate in the pilot study.
Intervention: The online system, Mobile Fact and Concept Training System (MoFaCTS) is an adaptive personalized learning system for content-area reading. MoFACTS uses a predictive mathematical model to sequence questions for optimal learning, based on principles of memory, such as the spacing effect, and principles of coherence, such as contiguity. In the MoFACTS system, an instructor approves an assignment, the system generates a website link for students to complete it, and the student begins a session after reading the assigned text. Researchers will further refine MoFACTS so that it provides students with an adaptive sequence of cloze items (sentences where a word or phrase is missing) that are automatically generated from course textbooks. When students answer questions correctly, MoFACTS is less likely to ask them the question again, and when students answer the question incorrectly, it will launch an auto-generated tutorial dialog to help students construct an explanation. In addition to this functionality, the researchers will also develop teacher tools so that instructors can better track student progress, choose which generated items to use, add their own items, and assign tasks.
Research Design and Methods: The iterative development will take place across three phases: system building, refinement, and pilot testing. During the system building phase, the researchers will focus primarily on developing the authoring, dialog, and cloze items modules, gather feedback from teachers and outside experts on the content that MoFACTS auto-generates and the functionality of the system. They will conduct small design studies to gather student and teacher feedback on the usability and feasibility of the system. During the refinement phase, they will integrate feedback, expand the question and tutorial dialog item base, and gather more user input to further refine the system. Once they have a fully operational MoFACTS, they will begin the pilot phase. The pilot study leverages a delayed treatment design with randomization within instructors at the classroom level. No classroom will use MoFACTS for the first two course exams, treatment classes will begin using after the second exam and throughout the remaining six exams, and the control classes will begin using it after the fourth exam.
Control Condition: The delayed treatment group serves as the comparison for between-group comparisons.
Key Measures: For the usability and feasibility studies, the researchers will develop their own survey measures, building off of existing work, such as the Motivated Strategies for Learning Questionnaire and the Technology Acceptance Model. For the pilot studies, they will also include student exam grades.
Data Analytic Strategy: The researchers will use hierarchical linear modeling and will use regression-based approaches in the adaptive practice scheduling algorithm and to model student growth.
Cost Analysis: The researchers will estimate costs for MoFACTS by considering multiple factors including the level of distribution (single class, school, multiple schools), personnel, training, and cost for a cloud-based server or alternative equipment for non-cloud deployment. They will estimate cost effectiveness as at the ratio of the cost of the intervention to the effect size with respect to learning gains, as estimated through exam scores.
Related IES Projects:
Hu, X., Cai, Zhiqiang, & Olney, A. M. (2019). Semantic Representation and Analysis (SRA) and its Application in Conversion-Based Intelligent Tutoring Systems (CbITS). In R. Feldman (Ed.), Learning Science: Theory, Research, and Practice (pp. 103–126). McGraw-Hill Education.
Graesser, A. C., Greenberg, D., Olney, A., & Lovett, M. W. (2019). Educational Technologies that Support Reading Comprehension for Adults Who Have Low Literacy Skills. In The Wiley Handbook of Adult Literacy (pp. 471–493).
Journal article, monograph, or newsletter
Eglington, L. G., & Pavlik Jr, P. I. (2019). Predictiveness of Prior Failures is Improved by Incorporating Trial Duration. JEDM| Journal of Educational Data Mining, 11(2), 1-19. Full text
Eglington, L. G., & Pavlik Jr, P. I. (2020). Optimizing practice scheduling requires quantitative tracking of individual item performance. NPJ science of learning, 5(1), 1-10.
Pavlik Jr, P. I., Olney, A. M., Banker, A., Eglington, L., & Yarbro, J. (2020). The Mobile Fact and Concept Textbook System (MoFaCTS). In CEUR workshop proceedings (Vol. 2674).