Assessing the use of automatic speech recognition and large language models for individuals with language impairments
TL;DR
This work systematically evaluates edge-oriented ASR-LLM pipelines for individuals with language impairments using comparison studies and ablation experiments across aphasia, child language impairment, and dementia datasets to identify transcript errors, repetition, noise, and input length as factors affecting system performance and deployment feasibility.
Abstract
Large language models (LLMs) have attracted much attention for healthcare applications, demonstrating strong potential in automating conversational interactions. However, cloud-hosted systems raise privacy concerns and text-based interfaces can be difficult for children, older adults, and individuals with language impairments. Edge-deployed, voice-enabled LLM systems may address these barriers by keeping data local and using automatic speech recognition (ASR) for speech input. However, ASR transcripts often contain disfluencies, fillers, grammatical errors, and recognition noise, which may reduce downstream LLM performance for speakers with atypical language patterns. Here, we systematically evaluate edge-oriented ASR-LLM pipelines for individuals with language impairments using comparison studies and ablation experiments across aphasia, child language impairment, and dementia datasets. We identify transcript errors, repetition, noise, and input length as factors affecting system performance and deployment feasibility. These findings highlight key challenges for building more reliable and accessible speech-enabled AI systems for healthcare applications.