AI-Driven Formative Assessments for Hands-on Science

During Phase I in 2020 the team developed a prototype of TaylorAI, including an AI algorithm and training data set, and a physical science activity for generating, analyzing, and interpreting graphs of motion to investigate relationships between variables, slope, and velocity. Researchers conducted a pilot study with six middle school science teachers and 120 students to assess the usability, feasibility, and promise of the prototype to offer feedback on the experiments to support student learning. Results demonstrated that the prototype AI mechanism generated results that students and educators were able to view, and student self-reported that the prototype increased their ability to correctly analyze graphs generated from their own experiments. Teachers reported that students were engaged by and used the formative results generated by the prototype.

During Phase II, the team developed a prototype of TaylorAI to provide automated formative feedback to students for eight physical science activities covering middle school physical science NGSS performance expectations. Researchers conducted a study with 19 middle school science classrooms to assess the formative assessment features. Researchers developed a 19-question assessment that was administered both before and after their exposure to PocketLab TaylorAI. The content of the assessment comprised selected items adapted from standardized assessment released items and was specifically aligned with the NGSS waves, energy, and gravity content covered in the TaylorAI science investigations. Qualitative feedback from teachers and students showed promise for how active science investigations can increase student motivation. Teacher and student feedback and classroom observation data will be incorporated into the refinement of the overall platform and the specific TaylorAI features. Quantitative results were inconclusive due to the short study duration and the breadth of science content involved. The quasi-experimental impact analysis had limitations due to the small sample size.