Skip Navigation

Success Stories out of the Institute of Education Sciences' SBIR Program: Analytic Measures, Inc. (AMI)

Screenshot of Analytic Measures

Product: Moby.Read® is a fully automated Oral Reading Fluency (ORF) assessment for students in Kindergarten to Grade 5. Moby.Read, which is fully student self-administered, replaces face-to-face assessments conducted by teachers, thereby saving teachers the time of having to assess each student individually.

SkillCheck is a component of Moby.Read that employs natural language processing software to analyze recordings of oral reading renditions of text passages and produce interactive report pages that rate and illustrate the student's basic reading skills as analyzed from recorded read-aloud performances.

The Moby.Read app includes 72 narrative and informational passages that cover 18 grade-leveled benchmark assessments and presents stories for students to read aloud, retell the stories, and answer questions aloud about the content. The app records and scores everything the student says and immediately reports ORF scores—reading level, word accuracy, accuracy rate (words correct per minute), comprehension, and expression. In addition, scores and audio recordings can be shared with reading specialists, parents, and students. A teacher dashboard shows assessment results at the individual and class levels and gives each teacher detailed reports on the reading strengths and difficulties of each student, and through the SkillCheck component presents recommended practice activities appropriate for the level of individual students. Moby.Read can be implemented in or out of classrooms to support oral reading instruction and encourage improved fluency. To support use and implementation, Moby.Read includes a Technical Manual and an Installation, User Management, Test Administration, and Scoring Guide.
Screenshot of MobyRead book reading platform for children

Research & Development: Prior to the ED/IES SBIR-supported projects, the developers who founded Analytics Measures, Inc. (AMI) designed several early prototypes of ORF assessments as part of awards made by the National Center for Education Statistics. For the 2002 NAEP ORF study, AMI built a laptop administered ORF with a scoring platform. For the 2004 Fluency Addition to NAAL, the system was designed to automatically score oral reading responses. For a 2014 contract, AMI prototyped an assessment app to score oral readings in classroom settings. And for the 2018 eNAEP, AMI implemented another ORF assessment to score students' read-aloud performances.
Screenshot of MobyRead Scoring Platform

The purpose of the ED/IES SBIR awards to AMI was to fully develop the Moby.Read, a student self-administered ORF to be used in school settings or at home. Moby.Read was designed with standards-aligned reading passages for ORF benchmarking, combining automatic speech recognition and refined natural language processing technology to automatically score reading level, words correct per minute, comprehension, and expression. To make classroom administration feasible, Moby.Read incorporates a student-interface for fully automated self-administered assessments. For teachers, the dashboard immediately presents texts, recordings, and results. During the project, AMI iteratively refined Moby.Read to ensure that all new features are clear and useful for teachers, and that the assessment flow is intuitive—even for early elementary students.

Once development was complete, several studies were conducted to test the usability, feasibility, reliability, and validity of Moby.Read. To test usability and classroom feasibility, seven teachers administered the Moby.Read assessment to their students. The teachers agreed that the visual appearance and automated administration was useful, that the app is intuitive for students, and that it provided appropriate content. In addition, 19 of 20 students reported that they preferred self-administering the assessment compared to being tested by a teacher. In another study, 304 grade school students completed grade-level appropriate forms of both Moby.Read (self-administered and automatically scored) and DIBELS (teacher-administered and teacher-scored). Results showed high correlations between the overall ORF scores and the automatic words correct per minute values, demonstrating that the technology-delivered assessment produced consistent results that align closely with a gold-standard paper assessment and that Moby.Read can be a viable replacement for traditional methods of teacher-administered ORF.

After the Moby.Read SBIR project ended in 2019, AMI conducted research to extract more detailed and diagnostic information from passage read-alouds to discover how well automated methods can isolate and measure foundational reading skills from passage readings. In 2019, AMI conducted a feasibility study to understand how this information can be used to guide instruction. Five reading specialists produced 1,813 free-response observations and suggestions from listening to 60 students in Grades 1 to 4 reading 27 passages. Results showed that six new machine measures outperformed all 18 of the specialist-assigned observations in predicting recommended suggestions.

In 2020 and 2021, AMI was funded by ED/IES SBIR to develop SkillCheck, a component that is integrated within Moby.Read that employs natural language processing software to analyze recordings of oral reading renditions of text passages and produces interactive report pages that rate and illustrate the student's basic reading skills as analyzed from recorded read-aloud performances. SkillCheck presents strengths and weaknesses and recommends practice activities appropriate for the level of individual students. Pilot research with 165 students in grades one to four demonstrated that the prototype of SkillCheck functioned as expected, that students were able to complete one oral reading task independently, and that the prototype generated diagnostic and formative information to students on their strengths and areas to improve their oral reading.

Path to Commercialization: Since AMI commercially launched Moby.Read in 2019, students across 30 states have taken over 30,000 ORF assessments. AMI's commercialization strategy includes both direct sales and partnerships with curriculum and assessment publishers. In 2019, K12 Inc. began using Moby.Read as an embedded benchmark assessment in their digital library, Big Universe, and adapted AMI's ORF technology to score shorter, formative assessments. BBB, a distributor in the Southeast, is reselling Moby.Read in Georgia, Florida, and Alabama, alongside other literacy products from larger publishers. In 2020, AMI announced a partnership with University of Oregon's Behavioral Research and Teaching Unit (BRT) to license reading passage content and machine learning annotations for Moby.Read. AMI also collaborated with Savvas Learning Company to integrate Moby.Read within Savvas's digital library and core literacy program. AMI used social media channels such as LinkedIn, Twitter, and Facebook to build the Moby.Read brand and attract district customers, and to support distribution partners.

On April 1, 2022, Google acquired the Intellectual Property rights for Moby.Read and SkillCheck in order to integrate these technologies within their suite of educational products. AMI will continue as a small business and is honoring school contracts that use Moby.Read and Skill Check until 2024.

Awards & Recognition:

2020: Finalist for an SIIA Codie Award

Peer Reviewed Publications:

  • Bernstein, J., Cheng, J., Balogh, J., & Downey, R. (2020). Artificial intelligence for scoring oral reading fluency. In H. Jiao & R. Lissitz (Eds.), Applications of artificial intelligence to assessment. Charlotte, NC: Information Age Publisher. View Chapter
  • Cheng, J. (2018). Real-time scoring of an oral reading assessment on mobile devices. Proceedings of Interspeech 2018, 1621–1625.View Article