Kasi Learning System: Accessible Science Diagrams through Multisensory Augmented Reality

In the prior 2021 Phase I project, the team developed a prototype of an augmented reality system that provides audio generated information on scientific images and diagrams used for teaching and learning chemistry. The prototype, called Kasi, also included tactile physical manipulatives and software to provide students a more comparable experience of the visualization to that of their sighted peers. At the end of Phase I, a pilot study was conducted with three special education teachers and 10 high school students who are blind or visual impaired. Results demonstrated that the prototype functioned as planned, all students reported that they were engaged and learned chemistry concepts, and educators believed that the full product would be feasible to integrate into classroom practice.

To fully develop Kasi in Phase II, the team will add chemistry content, optimize the calibration system to accommodate differing camera angles and lighting conditions, and finalize the digital interactive pedagogy, tactile pieces, and algorithms that generate audio feedback. After development concludes, researchers will conduct a pilot study to assess the feasibility and usability, fidelity of implementation, and promise of the product to support high school students who are blind or visual impaired in learning chemistry. The project team will collect data from a new group of 10 high school students who are blind or visually impaired to examine whether they can use the product independently and their chemistry content knowledge is improved. Chemistry knowledge will be measured by items from the American Chemical Society (ACS) High School exam and state assessments. Researchers will also conduct a cost analysis using the CostOut tool to determine the cost per student needed to implement and use Kasi.