Company Website: www.literably.com

Video Demonstration of the Phase I Prototype:  https://www.youtube.com/watch?v=QOIWlPTFbNs

Purpose: This project team will develop an oral reading fluency assessment for students in Kindergarten to Grade 5. Today, schools assess students across a range of critical literacy skills to identify instructional needs. Many of these skills are best assessed orally, but computers are still unable to process speech with human accuracy.

Project Activities: In the Phase I project in 2021, the research team developed a prototype to assess early grade school students' oral reading accuracy, including their rate and comprehension when reading aloud. The prototype prompted students to complete tasks by reading a word aloud into a device, and after the automatic speech recognition engine and a human rater scored students' oral responses.At the end of Phase I, the research team completed a pilot study with 22 educators and 450 students in Grades 1 to 3 that found that the prototype functioned as intended as students were able to self-administer the tasks and over 75% of teachers reported that the results of the assessment were accurate and helpful for informing instructional practice.

During Phase II development, the team will focus on optimizing the back-end data infrastructure, expanding the assessment tasks, and finalizing the user interfaces for students and educators. After development is complete, researchers will conduct a pilot study with 48 Kindergarten to Grade 5 classrooms (eight per grade). Half of the classrooms will be assigned to use the treatment and half will follow the business-as-usual assessment of oral reading fluency. The study will examine the feasibility, usability, fidelity of implementation, and validity of the assessment, and its promise for improving oral reading outcomes from pre-test to post-test.  To measure the validity of the assessment, researchers will compare scores with validated measures of oral reading, such as the STAR Early Literacy, STAR Reading, and the mCLASS. Researchers will gather cost information using the "ingredients method" and will include all expenditures on items such as personnel, facilities, equipment, materials, and training.

Product: The project team will develop the Platform for Actionable Reading Assessment (PARA), a comprehensive oral literacy assessment to efficiently and accurately measure the components of reading, including rapid naming, phonological awareness, phonics, fluency, vocabulary, comprehension, and spelling. For oral tasks, PARA will prompt the student to complete the task aloud and will record their responses. A mixture of human transcription and automatic speech recognition will score the responses and, within 24 hours, post the scores to a teacher dashboard to inform instruction. The assessment will replace one or more existing assessments of oral reading fluency, and results will be shared with teachers so that they can integrate the assessment findings into formal in-class learning. Training resources will support teachers with implementation, including videos, slide decks, and help articles.

Related IES Project: Online Oral Phonological Awareness Assessment (91990021C0024)