Co-Principal Investigators: Feil-Seifer, David; Hadley, Pamela A.; Kientz, Julia A.; Xiong, Jinjun

Purpose: The purpose of this project is to advance artificial intelligence (AI) and create the technology to assist speech-language pathologists (SLPs) with identifying students in need of speech and language services and delivering individualized interventions. Students who experience difficulties with speech or language are at an increased risk of falling behind in their academic and social-emotional development. The COVID-19 pandemic has strained our school systems, which, coupled with a national shortage of SLPs who provide screening and intervention services for these students, further exacerbated the existing education gap between students with and without speech and language difficulties. It is estimated that more than 3.4 million children need speech and language related services in the U.S. school system, yet there are less than 6,100 SLPs. This shortage makes it almost impossible for SLPs to identify all students who may need services and provide individualized services for those who do need it. To address the increased need for early identification and intervention, the researcher team at the AI Institute for Exceptional Education (AI4ExceptionalEd) will advance the frontiers of AI and develop two technologies—the AI Screener to enable and improve early universal screening of all students for speech and language difficulties and the AI Orchestrator to support SLPs in providing evidence-based individualized interventions that are in alignment with each student's individualized education plan (IEP).

Project Activities and Products: The research team will work to advance AI technology using multiple cutting-edge AI methodologies and develop new technology for SLPs to use in education settings. The AI Screener will be developed to improve early identification of students' speech and language difficulties. The AI Orchestrator will be designed to help SLPs create and administer individualized speech and language interventions and will be evaluated to assess the effects on meeting children's individual IEP learning targets.

Structured Abstract

Research Setting: The research will take place in urban elementary schools in Western New York, Illinois, Pennsylvania, Texas, Nevada, Washington, Oregon, and California.

Sample: SLPs working with elementary school students will be involved in the development, validation, and efficacy testing of the new AI technological tools. During the development of the new AI tools, approximately 10 SLPs will be consulted and will test out the technology, along with a small number of students. During the validation phase of each tool, approximately 100 students will participate (half with and half without speech and language difficulties) along with the SLPs in these their schools. For the efficacy trial, approximately 480 students from kindergarten to grade 5 will participate along with the SLPs in their schools. Within each school, approximately four classrooms will participate, with about six students with speech and language difficulties per classroom. The children assigned to the control classrooms will receive their usual education and related services.

Intervention: The AI Screener will improve and assist with universal early screening by analyzing student data and identifying those who may need speech and language services and require further diagnostic evaluation by an SLP. Specifically, the AI Screener will analyze video and audio streams of students’ classroom interactions and derive measures of speech and language, including the use of complex analyses to provide insight into language processes that are otherwise difficult for humans to assess (such as the automaticity of speech production). The AI Orchestrator will support SLPs with creating and administering evidence-based interventions, continuously monitoring student learning progress, and recommending the most appropriate modifications to intervention based on an individual student’s learning.

Research Design and Methods: The research team will use a wide array of methodologies to advance core AI capabilities and create new technological tools. Specifically, multimodal learning, meta learning, reinforcement learning, deep learning, and graph neural networks, as well as use-inspired approaches such as social robotics, computer vision, and natural language processing, will be utilized to develop the AI innovations. The team will then work in partnership with SLPs to use this innovative technology to develop two new AI tools to support the learning of children with speech and language difficulties—the AI Screener and AI Orchestrator—and evaluate their validity and efficacy. For validation of the AI Screener, students with and without speech and language difficulties will be given various speech- and language-related measures (such as oral reading fluency) by the AI Screener. Manual scoring of the administration by an SLP will be compared to scores computed by the AI Screener. To validate the appropriateness of the selected intervention plan, the AI Orchestrator will conduct multiple intervention sessions individually with students. The sessions will be recorded and watched by multiple SLPs, who will independently evaluate the appropriateness of the intervention plans created for each child. Finally, a cluster randomized controlled trial with a matched design study will be conducted across one school year to determine the efficacy of the AI Orchestrator, using a battery of standardized student measures of early speech and language to examine student outcomes.

Key Measures: A battery of standardized assessments will be administered both for study inclusion criteria and to measure treatment outcomes, including the Clinical Evaluation of Language Fundamentals, Test of Narrative Language, Diagnostic Evaluation of Language Variation, Peabody Picture Vocabulary Test, Woodcock Reading Mastery Test, Preschool Test of Nonverbal Intelligence, and the Goldman-Fristoe Test of Articulation. Additionally, researcher-designed proximal measures will be created for target vocabulary understanding and social interactions.

Data Analytic Strategy: To determine the validity of the selected intervention, interrater reliability will be calculated to determine agreement between the technology and the SLPs. For the cluster randomized controlled trail, hierarchical linear modeling will be used to analyze treatment effects for each outcome using a three-level nested model (students, classrooms, and schools), controlling for student characteristics. These models will also be used to examine possible variations in the treatment effects among different schools.

* Total federal award from IES and NSF