Capti Assess: Personalizing Reading for Understanding Assessments with Adaptive Multistage Design

In the Phase I project in 2021, the team developed a prototype which included scenario-based reading assessments for students in Grades 3 to 12 that employ machine learning to personalize the user experience and a teacher dashboard that provides insights to inform instruction on the assessments. The Phase I pilot study with 50 students and four Grade 6 educators demonstrated that the prototype functioned as planned, that students were able to engage in performance-based reading tasks, and that teachers perceived that the reports on student performance were accurate and could be used to provide instruction.

In the Phase II project, the team will fully develop the product, including finalizing the content of the assessments, optimizing the user-experience and implementation supports, developing additional scenario-based assessments, and validating the machine learning algorithm to personalize the assessments based on students' level of proficiency. Iterative refinements will be conducted with feedback from educators and students at major production milestones until the product is fully functional. After development concludes, researchers will conduct a pilot study to assess the feasibility and usability, fidelity of implementation, and the promise of the product to assess and improve reading. The team will collect data from 240 students, including 80 each from elementary, middle, and high schools. Half of the sample will be randomly assigned to use the product and the other half will use business-as-usual reading assessment activities. Researchers will compare pre- and post-scores for students' scores on standardized reading measures, including the STAR, iReady, and a New York State standardized test. Researchers will gather cost information using the "ingredients method" and will include all expenditures on things such as personnel, facilities, equipment, materials, and training.