This work investigates parameter-efficient approaches for transferring knowledge from resource-rich source languages to low-resource target languages on Whisper, and proposes Sequential Adapter Stacking, which places a trainable target-language adapter on top of a frozen source-language adapter.
Abstract
Extending large-scale multilingual automatic speech recognition (ASR) models to low-resource languages remains challenging. Model performance is skewed toward high-resource languages and degrades sharply for languages with limited labeled data and pre-training exposure. To address this, we investigate parameter-efficient approaches for transferring knowledge from resource-rich source languages to low-resource target languages on Whisper. Alongside warm initialization and attention-based fusion, we propose Sequential Adapter Stacking, which places a trainable target-language adapter on top of a frozen source-language adapter. Under controlled experiments, these approaches are evaluated on three target languages unsupported by Whisper -- Asturian, Assamese, and Xhosa -- using source languages with varying degrees of relatedness. Sequential Adapter Stacking with the closest related source consistently and significantly outperforms full fine-tuning across the three targets, with 5--8\% relative WER reductions. These gains largely persist with only one hour of target training data.
This work combines vocabulary surgery for tokenizer adaptation without full retraining, cross-lingual transfer of quality classifiers via translation, enabling filtering without target-language annotations, and generation of instruction data through translation, task conversion, and targeted synthesis.
Yurii Paniv, Bohdan Didenko, Mykola Haltiuk et al.· 4 citations
Cross-lingual zero-shot transfer and multilingual fine-tuning are promising approaches for NLP tasks such as Named Entity Recognition (NER) in low-resource languages, but in the absence of target language benchmarks, it is unclear which auxiliary language selection strategy leads to the best transfer. We introduce Runy...
Prosper Arineitwe Asiimwe, Francois Meyer, Jan Buys· Proceedings of the 6th Works...· 0 citations
This work proposes an approach that integrates code-switching directly into masked language model pretraining, and introduces a multiview probabilistic translation strategy that samples candidate translations based on alignment likelihoods, applying substitutions only to unmasked tokens.
Ruan Visser, Trienko L. Grobler, Marcel Dunaiski· 0 citations
Centroid Intervention Fusion is proposed, a projection fusion framework that consolidates multiple multilingual intervention projections into a single language-shared operator and outperforms the strongest prior pairwise intervention baseline by up to +3.3% across four model backbones.
Multilingual Large Language Models (LLMs) traditionally rely on a single vocabulary shared by all supported languages, which can lead to uneven compression across them. Moreover, their large embedding and output matrices increase memory usage and slow inference, notably for small-scale models. It is also wasteful as mo...
Franck Signe Talla, Hippolyte Pilchen, François Yvon et al.· Conference on Empirical Meth...· 0 citations
This work proposes PSMC (Pre-train, Specialize, Merge, Merge, and Co-train), a data-efficient framework that capitalizes on a cross-script "transfer effect" to fuse language-specific experts into a unified, high-performance multilingual back- bone.
P. Achyuth, Kahaan Shah, Chetan Arora· 0 citations