Aug 2026· 2026 6th International Conference on Emerging Smart Technologies and Applications (eSmarTA)· pp. 1-8· 0 citations· 38 references
Abstract
Detecting hate speech in low-resource and unseen languages remains challenging due to limited labeled data and linguistic diversity. This paper presents a comparative study of zero-shot cross-lingual transfer for hate speech detection using two multilingual transformer models: mDeBERTa-v3 and XLM-RoBERTa. To the best of our knowledge, mDeBERTa-v3 has not been previously used by researchers for zero-shot cross-lingual hate speech detection, making this the first study to evaluate its capabilities in this task. Furthermore, we introduce new unseen languages that have not been studied before in this context, including Hebrew, Amharic, and Swahili, alongside other languages such as Indonesian, Danish Portuguese, Turkish, French, and Russian. We evaluate model performance under three training scenarios: a single source language (Turkish), semantically similar language clusters, and multiple clusters from different language families. Experimental results show that mDeBERTa-v3 consistently outperforms XLM-RoBERTa in zero-shot settings. The most notable improvement is observed for Hebrew, where the macro F1 score increases from 0.39 (XLM-RoBERTa) to 0.71 (mDeBERTa-v3), a gain of 0.32. Substantial gains are also seen for Amharic (0.52 → 0.73, +0.21), Indonesian (0.57 → 0.71, +0.14), and Swahili (0.65 → 0.75, +0.10). Across all experimental conditions, mDeBERTa-v3 achieves average macro F1 gains ranging from 0.04 to 0.19, with statistical significance (p < 0.02). The model’s advantage is attributed to its disentangled attention mechanism, which enables better generalization across typologically distant languages. These findings establish mDeBERTa-v3 as a novel and more robust architecture for zero-shot cross-lingual hate speech detection, particularly for previously unexplored low-resource languages.
It is challenging to detect hate speech in Low Resource Languages (LRLs) because of the absence of annotated data, the informality of its language structure, and the lack of standardized grammar. A good example of such a challenge is Roman Urdu which is broadly used by South Asians on social media and has a high variation while lacking contextually consistent spellings. The objective of this paper is to conduct a comprehensive assessment of Large Language Models (LLMs) for Hate Speech Detection (HSD) in Roman Urdu script and fine-tune these models using the Parameter-Efficient Fine-Tuning (PEFT) method called Low-Rank Adaptation (LoRA). To evaluate zero-shot inference, we benchmarked it against PEFT on different transformer models, including Mistral, LLaMA, Falcon, and multilingual BERT. Experiments are conducted on the PURUTT (Parallel Urdu and Roman Urdu Corpus for Toxic Comments and Transliteration) dataset with over 72,000 annotated comments. The results suggest that zero shot models perform moderately (F1 = 0.56), but updating a small fraction of the model trainable parameters improves the classification performance significantly (F1>0.93). Our results have shown that PEFT delivers outstanding performance alongside excellent computational efficiency, making it highly suitable for low-resource language processing tasks.
Toneema Zubair, Muhammad Asif, F. Kamiran et al.· 0 citations
Findings indicate that language-specific fine-tuning plays a more critical role than multilingual generalization in achieving accurate ASR for Indonesian and provide practical guidance for deploying ASR systems in low-resource language scenarios.
J. Hebert, Amalia Zahra· Bulletin of Electrical Engin...· 0 citations
A multilingual zero-shot TTS system that supports 14 languages and performs both intra-lingual and cross-lingual reference cloning without requiring transcripts of audio prompts, which achieves the best average overall rank in human evaluation among recent open-source and commercial systems.
Hua-Xuan Wang, Hui-Min Wang, Ruiyu Zhang et al.· 0 citations
Cross-lingual mismatch remains a key source of overall degradation in modern speaker verification. The TidyVoice2026 Challenge targets this setting with text-independent verification, comprising 3,666 training and 808 development speakers in 40 languages and 2,200 evaluation speakers in 38 unseen languages, without language labels at test time. Starting from the official SimAM-ResNet34 baseline pretrained on VoxBlink2 and VoxCeleb2 and fine-tuned on TidyVoice, we revisit Nuisance Attribute Projection (NAP) as a simple language-normalization step in the embedding space. We estimate a compact language subspace from cross-language same-speaker differences and project embeddings onto its orthogonal complement before cosine scoring with Adaptive Symmetric score normalization. This reduces development EER from 2.97\% with cosine and 2.70\% with AS-Norm to 2.18\% and yields a Codabench evaluation score of 8.40, showing that simple back-end language normalization can rival more complex systems.
We present DONDO, a family of open, permissively licensed automatic speech recognition (ASR) base models for African languages, built on the w2v-BERT 2.0 self-supervised speech encoder. DONDO comprises twenty-one monolingual models and five multilingual models spanning twenty-seven language varieties across Ghana, Sierra Leone, Nigeria, Senegal, Kenya and Zimbabwe. Models are fine-tuned primarily on read speech drawn from religious texts, which offer broad, license-clear and orthographically consistent coverage for languages that otherwise lack transcribed audio. We describe a two-step (and, for one family, three-step) learning-rate-annealed fine-tuning procedure that first adapts a shared multilingual model at a high learning rate and then anneals it to recover, and in several cases surpass, strong monolingual baselines. We further describe a lightweight language-conditioning mechanism that injects a one-hot language identity as a sequence of prefix frames prepended to the acoustic features, allowing a single multilingual checkpoint to be steered to a target language at inference. Across the five multilingual families the annealed models reach average word error rates (WER) of 10-13%, closing most of the gap to monolingual models while covering many languages in a single checkpoint. All models are released on the Hugging Face KhayaAI organisation under the Apache-2.0 license (attribution only) so that others may fine-tune them freely, including for commercial use. We provide a conservative estimate that the languages covered are spoken by on the order of one hundred million first-language speakers, and by substantially more when second-language use is included.
P. Azunre, Naafi Dasana Ibrahim, Joel Budu et al.· arXiv.org· 0 citations
This study explores a low-resource approach to detecting hate speech in English and Swahili code-switched text by fine-tuning pre-trained language models, and shows that fine-tuning modern language models can offer a practical and scalable solution for hate speech detection in multilingual environments.
Kipkebut Andrew, Jepkemei Betty· International Journal for Re...· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.
