This work introduces LipCoder, a voice-centric programming toolkit designed to deliver editor-level functionality through auditory and speech-based interfaces, and quantitative trends and qualitative feedback point to directions for auditory-first design that may broaden accessibility in speech-driven coding environments.
Abstract
AI-assisted programming environments have accelerated software development, giving rise to new paradigms like vibe coding. However, their benefits remain largely inaccessible to visually impaired programmers, as existing screen readers and assistive tools offer limited support for these emerging workflows. We introduce LipCoder, a voice-centric programming toolkit designed to deliver editor-level functionality through auditory and speech-based interfaces. LipCoder offers features comprising speech feedback and earcon cues for comprehension and validation, as well as natural language input for navigation and modification. In an exploratory evaluation, 5 visually impaired programmers performed a series of coding tasks comparing LipCoder with a baseline of VSCode, Copilot, and VoiceOver. Quantitative trends and qualitative feedback point to directions for auditory-first design that may broaden accessibility in speech-driven coding environments.
As multimodal interaction becomes central to digital learning, technical interfaces, such as eBooks and conversational agents, present unique accessibility challenges for users relying on the auditory modality. While modern Text-to-Speech (TTS) systems excel at rendering natural language, they often fail to convey the...
Marco Cardia, Letizia Angileri, M. Buzzi et al.· 0 citations
Audiodescription is essential for making audiovisual content accessible to visually impaired individuals, converting visual information into audio descriptions. Typically done by human narrators, audiodescription can be complemented by virtual technologies with synthesized voice. This study evaluated the accessibility...
A. D. de Oliveira· Revista Brasileira Multidisc...· 0 citations
Speech editing is gradually making its way into everyday applications. This paper presents Speech Editor, a practical software system built upon SpeechTripleNet for disentangled speech representation learning. The system enables real-time end-to-end speech editing, including local modification of pitch and energy as we...
Jiayi Wang, Qifeng Han, Danyi Huang et al.· International Conference on...· 0 citations
An interpretable audio-visual synchronization framework that explicitly models speech-lip temporal alignment through feature encoding, cross-attention fusion, lag estimation, stability quantification, and visualization is proposed.
Bo-Wen Yu, Ming-Yue Huang, Yi-Shen Liu et al.· 0 citations
Software debugging is a crucial but lengthy process in software development that may require developers to decode complicated error messages and stack traces. The growing complexity of Python applications today underscores the need for more intuitive and effective debugging methods. This paper focuses on enhancing debu...
P. L., Rakshitha S, M. P et al.· Engineering & Technology· 0 citations
Environmental noise obscures the content cues needed for whisper-to-normal conversion. We propose WhisperVC-AV, which restores acoustic content features while leaving the original WhisperVC conversion module unchanged. Its context-guided restoration module combines temporal acoustic context with synchronized lip featur...
Zi-Yue Yin, Dong Liu, Ming Li· 0 citations
Related blog posts
MIT News · Artificial Intelligence· news.mit.eduOct 28, 2026
Professor Paris Smaragdis develops computers with superhuman hearing for applications like speech recognition, music understanding, and environmental monitoring.
MIT News · Artificial Intelligence· news.mit.eduOct 23, 2026
During the School of Humanities, Arts, and Social Sciences’ 75th Anniversary Symposium, President Sally Kornbluth said “MIT has never needed your particular insight and guidance more than we do now.”
MIT News · Artificial Intelligence· news.mit.eduOct 23, 2026
Assistant Professor Benjamin Lindquist studies how engineers deliberately injected randomness into early AI, giving rise to deep-learning models that sometimes hallucinate.
MIT News · Artificial Intelligence· news.mit.eduOct 22, 2026
“X-Hinges” helps design and 3D print items like lamps and gloves with an interactive twist: automatically embedded sensors that perceive and adaptively respond to human contact.
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.
