Structured Abstract
Setting
The research will take place in urban elementary schools in Western New York, Illinois, Pennsylvania, Texas, Nevada, Washington, Oregon, and California.
Sample
SLPs working with elementary school students will be involved in the development, validation, and efficacy testing of the new AI technological tools. During the development of the new AI tools, approximately 10 SLPs will be consulted and will test out the technology, along with a small number of students. During the validation phase of each tool, approximately 100 students will participate (half with and half without speech and language difficulties) along with the SLPs in these their schools. For the efficacy trial, approximately 480 students from kindergarten to grade 5 will participate along with the SLPs in their schools. Within each school, approximately four classrooms will participate, with about six students with speech and language difficulties per classroom. The children assigned to the control classrooms will receive their usual education and related services.
Intervention
The AI Screener will improve and assist with universal early screening by analyzing student data and identifying those who may need speech and language services and require further diagnostic evaluation by an SLP. Specifically, the AI Screener will analyze video and audio streams of students' classroom interactions and derive measures of speech and language, including the use of complex analyses to provide insight into language processes that are otherwise difficult for humans to assess (such as the automaticity of speech production). The AI Orchestrator will support SLPs with creating and administering evidence-based interventions, continuously monitoring student learning progress, and recommending the most appropriate modifications to intervention based on an individual student's learning.
Research design and methods
The research team will use a wide array of methodologies to advance core AI capabilities and create new technological tools. Specifically, multimodal learning, meta learning, reinforcement learning, deep learning, and graph neural networks, as well as use-inspired approaches such as social robotics, computer vision, and natural language processing, will be utilized to develop the AI innovations. The team will then work in partnership with SLPs to use this innovative technology to develop two new AI tools to support the learning of children with speech and language difficulties—the AI Screener and AI Orchestrator—and evaluate their validity and efficacy. For validation of the AI Screener, students with and without speech and language difficulties will be given various speech- and language-related measures (such as oral reading fluency) by the AI Screener. Manual scoring of the administration by an SLP will be compared to scores computed by the AI Screener. To validate the appropriateness of the selected intervention plan, the AI Orchestrator will conduct multiple intervention sessions individually with students. The sessions will be recorded and watched by multiple SLPs, who will independently evaluate the appropriateness of the intervention plans created for each child. Finally, a cluster randomized controlled trial with a matched design study will be conducted across one school year to determine the efficacy of the AI Orchestrator, using a battery of standardized student measures of early speech and language to examine student outcomes.
Key measures
A battery of standardized assessments will be administered both for study inclusion criteria and to measure treatment outcomes, including the Clinical Evaluation of Language Fundamentals, Test of Narrative Language, Diagnostic Evaluation of Language Variation, Peabody Picture Vocabulary Test, Woodcock Reading Mastery Test, Preschool Test of Nonverbal Intelligence, and the Goldman-Fristoe Test of Articulation. Additionally, researcher-designed proximal measures will be created for target vocabulary understanding and social interactions.
Data analytic strategy
To determine the validity of the selected intervention, interrater reliability will be calculated to determine agreement between the technology and the SLPs. For the cluster randomized controlled trail, hierarchical linear modeling will be used to analyze treatment effects for each outcome using a three-level nested model (students, classrooms, and schools), controlling for student characteristics. These models will also be used to examine possible variations in the treatment effects among different schools.
Supplemental information
Co-Principal Investigators: Feil-Seifer, David; Hadley, Pamela A.; Kientz, Julia A.; Xiong, Jinjun
Project Activities and Products: The research team will work to advance AI technology using multiple cutting-edge AI methodologies and develop new technology for SLPs to use in education settings. The AI Screener will be developed to improve early identification of students' speech and language difficulties. The AI Orchestrator will be designed to help SLPs create and administer individualized speech and language interventions and will be evaluated to assess the effects on meeting children's individual IEP learning targets.
* Total federal award from IES and NSF
Questions about this project?
To answer additional questions about this project or provide feedback, please contact the program officer.