Skip to content

Language Discrimination Improves Linguistic Learning in Multilingual Speech Models

Sep 2026 · arXiv.org · Vol abs/2609.33345 · 0 citations · 19 references
Computer Science Engineering

TL;DR

It is shown that strengthening the model's ability to discriminate languages during pretraining reduces and, on some measures, closes this multilingual gap on continuous phonetic and higher-level linguistic measures, while preserving substantial cross-language sharing.

Abstract

Multilingual self-supervised speech models can benefit from sharing information across languages, but under a matched total pretraining data budget they still fall short of monolingual models. We show that strengthening the model's ability to discriminate languages during pretraining reduces and, on some measures, closes this multilingual gap on continuous phonetic and higher-level linguistic measures, while preserving substantial cross-language sharing. Using a controlled English/French HuBERT setting, we test two interventions which strengthen language discrimination: an auxiliary language classifier and per-language k-means targets. Across interventions, continuous-feature phone discrimination error (phone-ABX, lower is better) decreases from 11.6% in the bilingual baseline to 10.4% (monolingual: 10.8%), while lexical performance (sWUGGY, higher is better) increases from 52.1% to 56.7% (monolingual: 58.5%) and prosodic performance (ProsAudit, lexical subtask, higher is better) from 68.9% to 72.9% (monolingual: 72.6%). Across HuBERT training stages, the strongest gains on most linguistic measures occur when language discrimination is introduced in the first iteration, whereas later or repeated interventions yield smaller improvements and are accompanied by increased language-wise segregation. These results support a causal role for language discrimination in reducing the additional cost of multilingual learning.

View source

Similar papers

#natural language process... Open access Sep 2026

Cross-Linguistic Effects in Bilingual Phoneme BabyLMs

Cross-linguistic effects are a central topic in bilingual first-language acquisition. Artificial learners can help investigate L1-L2 interactions by enabling controlled comparisons across language combinations and learning conditions. Recent work explores this direction by training bilingual language models under devel...

Nikitas Theodoropoulos, Maria Lymperaiou, Giorgos Filandrianos · 0 citations
#natural language process... Conference Open access Sep 2026

Assessing the Impact of Language Disparity on Multilingual Linguistic Ability in Large Language Models

Claims about the grammatical competence of multilingual language models vary sharply with how competence is measured, yet the interaction between evaluation paradigm, post-training, and language resource availability has not been systematically examined. We evaluate base and post-trained models from six families on Mul...

Zhan-Yu Chen, Jaap Jumelet · 0 citations
#natural language process... Open access Sep 2026

Turning Speech Language Models into Multilingual Listeners

MULTISPEECHQA, a large-scale, synthetically generated and human-verified dataset comprising 9200 hours of 10.8 million spoken question-answer pairs in 23 typologically diverse languages, is presented and shows that high-quality synthetic datasets offer a cheap solution to improving the multilingual capabilities of SLMs...

Tolúlopé Ògúnrèmí, D. Jurafsky, Christopher D. Manning et al. · 0 citations
#machine learning Preprint Oct 2026

Phonological Interference in Multilingual Speech Models

Phoneme-level models transcribe or generate speech as a sequence of phonemes, the smallest sound units that distinguish words. These models enable fine-grained pronunciation control and understanding, yet often fail on input that does not match any single training language, such as speech alternating between two langua...

Moran Yanuka, Raja Giryes, M. Alper · 0 citations
#artificial intelligence Preprint Oct 2026

Adapting English Quality Classifiers for Multilingual LLM Pretraining Data Selection

Recent advances in large language model (LLM) pretraining highlight the role of high-quality training data in improving performance. While model-based filtering has proven effective in selecting high-quality subsets from web-scale corpora, especially for high-resource languages, low-resource languages face challenges d...

Vinko Sabolcec, Bettina Messmer, Yassine Turki et al. · 0 citations

Related blog posts

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.