Company Website: www.captivoice.com

Video Demonstration of the Phase I Prototype: https://youtu.be/3HG77iXKAIk

Purpose: In this project, the research team will develop a scenario-based reading assessment that employs machine learning to personalize student learning. Students who fail to achieve proficiency in reading are at increased risk for dropping out of school. New forms of assessment can be used to measure student performance in real-time and to provide insights to educators to inform instruction.

Project Activities: In the Phase I project in 2021, the team developed a prototype which included scenario-based reading assessments for students in Grades 3 to 12 that employ machine learning to personalize the user experience and a teacher dashboard that provides insights to inform instruction on the assessments. The Phase I pilot study with 50 students and four Grade 6 educators demonstrated that the prototype functioned as planned, that students were able to engage in performance-based reading tasks, and that teachers perceived that the reports on student performance were accurate and could be used to provide instruction.

In the Phase II project, the team will fully develop the product, including finalizing the content of the assessments, optimizing the user-experience and implementation supports, developing additional scenario-based assessments, and validating the machine learning algorithm to personalize the assessments based on students' level of proficiency. Iterative refinements will be conducted with feedback from educators and students at major production milestones until the product is fully functional. After development concludes, researchers will conduct a pilot study to assess the feasibility and usability, fidelity of implementation, and the promise of the product to assess and improve reading. The team will collect data from 240 students, including 80 each from elementary, middle, and high schools. Half of the sample will be randomly assigned to use the product and the other half will use business-as-usual reading assessment activities. Researchers will compare pre- and post-scores for students' scores on standardized reading measures, including the STAR, iReady, and a New York State standardized test. Researchers will gather cost information using the "ingredients method" and will include all expenditures on things such as personnel, facilities, equipment, materials, and training.

Product: The team will fully develop Capti Assess, a scenario-based reading assessment that employs machine learning to personalize assessment for students in Grades 3 to 12.  At baseline, the product will evaluate reading skills using a 10-minute screenerthat evaluates comprehension, critical thinking, and perspective taking, and following a machine learning algorithm will route students to content aligned to their reading level. The content of the assessments are from existing evidence-based programs, including the ETS ReadBasix^TM (formerly RISE), and the Global, Integrated Scenario-Based Assessment (GISA), which was developed and evaluated through the  IES Reading for Understanding Research initiative.

Related IES Projects: Modernizing Reading Assessment with Scenario-Based Tasks (91990021C0029); Assessing Reading for Understanding: A Theory-based, Developmental Approach (R305F100005).