Skip to content
#natural language processing Conference Open access

SHAMS: An Audio-Grounded Pronunciation Benchmark for Levantine Arabic

Oct 2026 · Proceedings of The Fourth Arabic Natural Language Processing Conference · pp. 88-114 · 0 citations · 57 references
Computer Science

TL;DR

SHAMS (SHami Annotated Multi-dialect Speech), a benchmark comprising 1,300 utterances drawn from open audio corpora, balanced across five LA varieties, is presented, grounded in audio and stratified by variety.

Abstract

Levantine Arabic (LA) is spoken by tens of millions of people, creating a pressing need for shared benchmarks to evaluate LA speech-language technologies. Evaluating such technology is particularly challenging given LA's internal diversity and its opaque and non-standardized orthography. We present SHAMS (SHami Annotated Multi-dialect Speech), a benchmark comprising 1,300 utterances drawn from open audio corpora, balanced across five LA varieties (Urban and Rural Palestinian, and Urban Jordanian, Lebanese, and Syrian). Each utterance is represented across four aligned tiers: audio, unvocalized orthography, diacritized text, and phonetic transcription. This structure supports evaluation of various downstream tasks such as diacritization, grapheme-to-phoneme conversion, automatic speech recognition, and audio-to-phoneme, grounded in audio and stratified by variety. We benchmark open and proprietary models across these tasks to demonstrate the utility of this benchmark for measuring progress across LA. We release SHAMS at https://shams-nlp.github.io .

Read PDF

Similar papers

Almieyar: A Culturally Grounded Benchmark for Multi-Dialect Arabic Speech Recognition

ALMIEYAR is introduced, a culturally grounded ASR benchmark covering 17 Arabic dialects across six families, built entirely from newly recorded speech unseen by existing models, including the first published benchmark for Ahwazi Arabic.

Omid Ghahroodi, Anas Madkoor, Dima Faris Alsaudi et al. · 0 citations
#artificial intelligence Preprint Oct 2026

AraYoungVoices: A Diverse L1/L2 Corpus of Arabic Child and Adolescent Speech

State-of-the-art ASR systems primarily target native adult speech, leading to substantial performance gaps for children, adolescents, and L2 speakers. We introduce AraYoungVoices, a 151.72-hour Arabic read-speech corpus from 286 speakers aged 7--18, comprising AraKids (7--12) and AraTeens (13--18). The corpus includes...

S. Chowdhury, Zien Sheikh Ali, H. Lachemat et al. · 1 citation

CARDAMOM: A Micro-Dialectal Arabic Speech Dataset for ASR

Cardamom is presented, a micro-dialectal Arabic speech dataset designed to support fine-grained evaluation and adaptation of automatic speech recognition (ASR) systems, and the collection and annotation process is described, the micro-dialect inventory linguistically is motivated, and four multilingual ASR systems in z...

Bashar Talafha, Samar M. Magdy, Aisha Alansari et al. · 1 citation

VietPrism: A large-scale Vietnamese speech and deepfake corpus with diverse dialects and code-switching

Vietnamese speech research is constrained by resources that isolate automatic speech recognition from speaker, dialect, code-switching, and deepfake analysis. We introduce VietPrism, an open, multi-domain corpus that brings these dimensions together at scale: 993.4 hours and 403,941 bona fide utterances from 1,262 veri...

M. Hoàng, Thai Le · 0 citations
Open access Sep 2026

A pronunciation-sensitive analysis of XTTS adaptation for modern standard Arabic synthetic diacritization length rescue and training dynamics

The findings support a diagnostic rather than leaderboard interpretation: the apparent benefit of Arabic XTTS adaptation depends materially on the evaluation metric, checkpoint, and evaluator.

Abdennacer Elbasri, Abdennabi Morchid, Khalid Oqaidi et al. · 0 citations
Preprint Oct 2026

Pronunciation-Supervised Text-to-Speech for Irish (Gaeilge)

Speech technology for Irish must be built from very little transcribed speech, yet the language's pronunciation is thoroughly documented in dictionaries and dialect descriptions. We ask whether that documentation can supervise the adaptation of a large pretrained text-to-speech model to a language it was never trained...

William Lane · 0 citations

Related blog posts

MIT News · Artificial Intelligence Nov 17, 2026

Learning the language of machine learning

By studying the computational foundations of language learning, Associate Professor Jacob Andreas builds intelligent systems that can communicate effectively with humans.

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.