Aug 2026· Big Data and Cognitive Computing· 0 citations· 26 references
TL;DR
This work systematically compares zero-shot parametric generation, in-language retrieval, and cross-lingual (translate-then-retrieve) web search using three 4B-parameter SLMs in both reasoning and non-reasoning modes and proposes a training-free, self-aware router that uses majority voting over repeated self-verification decisions to determine when to search the web, and when to escalate to a more capable cloud model.
Abstract
Small Language Models (SLMs) enable efficient deployment, but their limited parameter count constrains factual knowledge, particularly in low-resource languages like Kazakh. Integrating live web search can address this limitation, though its effectiveness is difficult to measure due to sparse in-language web indices and answer leakage during benchmarking. In this study, we systematically compare zero-shot parametric generation, in-language retrieval, and cross-lingual (translate-then-retrieve) web search using three 4B-parameter SLMs in both reasoning and non-reasoning modes. To evaluate factuality without search-engine leakage, we introduce machine-translated Kazakh versions of the FreshQA and DefAn benchmarks, and use GPQA as a Google-proof adversarial control. We also assess robustness across three prompt complexities, from simple JSON constraints to adversarial warnings that instruct the model to treat potentially unreliable context with caution. Finally, we propose a training-free, self-aware router that uses majority voting over repeated self-verification decisions to determine when to answer parametrically, when to search the web, and when to escalate to a more capable cloud model. Our results show that cross-lingual retrieval substantially outperforms in-language search on global factuality tasks, nearly doubling accuracy on FreshQA, while direct in-language search remains preferable for localized cultural queries. The choice of retrieval language depends on the task and does not always favor English. Additionally, cross-lingual retrieval is not consistently superior, because the best option depends on where relevant information is indexed. Pareto analysis indicates that cross-lingual search is on or near the optimal accuracy–latency frontier, adding minimal overhead compared to direct search. The router identifies which query types warrant retrieval, and as a system it tracks or exceeds always-search accuracy while issuing fewer searches and approaching the always-cloud ceiling at a fraction of its cost; per-query discrimination within a task family is weaker, which we quantify explicitly. Overall, this work offers a framework for optimizing and accurately measuring cross-lingual RAG pipelines in low-resource settings.
Large language models (LLMs) are increasingly being used as automated judges for relevance evaluation in information retrieval, yet their robustness to adversarial manipulation remains insufficiently understood, particularly in multilingual settings. In this work, we investigate the impact of cross-lingual prompt injection attacks on LLM-based relevance judgments using TREC Deep Learning collections and two open-weight models under established prompting frameworks. We examine both instruction-based and content-based injection strategies in 8 languages spanning different resource levels. Our results demonstrate that multilingual query-based injections are highly effective in inflating relevance scores while simultaneously evading existing prompt-injection defenses. We further found that, although existing defense mechanisms can be modified to mitigate such attacks, these injections can be easily adapted to bypass them. These findings highlight a critical gap in current defense approaches and demonstrate that language generalization can act as an attack vector, underscoring the need for more robust and proactive evaluation frameworks for LLM-as-a-judge systems.
ABSTRACT
DocuMind is a fully offline, privacy-preserving, multilingual Document Question Answering system built on the Retrieval-Augmented Generation (RAG) architecture. The system enables users to upload documents in any language and ask questions in their native language — including Kannada, Hindi, Tamil, Telugu, and English
— receiving accurate, contextually grounded answers from a locally running Large Language Model (LLM) with zero cloud dependency and zero recurring cost. The system employs Language-Agnostic BERT Sentence Embeddings (LaBSE), a multilingual model supporting 109 languages, to map all text into a shared 768-dimensional semantic vector space, enabling true cross-lingual retrieval without any translation step. Document embeddings are persisted in a local ChromaDB vector database and answer generation is performed by LLaMA
3.2 (3B parameters) running locally via the Ollama runtime. Evaluation using RAGAS-style metrics across five languages demonstrates Faithfulness scores of 0.77 to 0.87 and average query latency under 8 seconds on a standard consumer laptop with 8 GB RAM.
KEYWORDS Retrieval-Augmented Generation, Large Language Models, Multilingual NLP, LaBSE, ChromaDB, Ollama, LLaMA, Document Question Answering, Indic Languages, Offline AI, Natural Language Processing, Cross-lingual Retrieval.
Swetha M S, Dr. Sripalreddy· International Scientific Jou...· 0 citations
We present L3Cube-IndicQuest v2, a large-scale gold-standard multilingual question-answering benchmark for evaluating the India-specific factual knowledge of Large Language Models (LLMs). The benchmark comprises 3,471 curriculum-grounded English question--answer pairs spanning nine domains, curated from educational curricula, competitive examination materials, and domain-specific reference books. We introduce a practical hybrid construction strategy that combines context-grounded LLM-based question generation and validation with semantic deduplication and human verification, enabling scalable creation of benchmark data while preserving annotation quality. The benchmark is translated into 19 Indic languages, yielding a publicly released multilingual dataset of 69,420 question--answer pairs across 20 languages. We evaluate six LLMs under three protocols: LLM-as-a-judge and two deterministic lexical criteria, exact-substring and word-overlap matching. All three produce almost the same model ranking, showing that the results do not depend on the choice of judge. The frontier commercial model leads by a wide margin, and among open-weight models Gemma4 31B outperforms the Indic-specialised Sarvam 30B in every evaluated Indic language.
Rinit Jain, Tirthraj Mahajan, Advait Joshi et al.· 0 citations
Safety alignment in large language models (LLMs) is largely developed in English, assuming these safeguards generalize across multilingual settings. However, this assumption remains underexplored and exposes a vulnerability in low-resource languages. We investigate cross-lingual safety transfer in four African languages, Twi, Hausa, Amharic, and Swahili, using LoDNA, a new safety dataset that pairs literal translations with culturally localized prompts. To move beyond generation-based evaluation, we propose a latent geometric framework that probes hidden-state refusal representations in LLMs. Our experimental results show that cross-lingual safety transfer is severely limited; harmful prompts retain less than 10% of the English refusal signal across most language-model pairs. Literal and localized prompts are semantically aligned (cosine 0.95-0.996) but drift across layers, suggesting models encode the concepts without routing them to safety mechanisms. These findings demonstrate that current multilingual safety alignment is superficial, providing strong evidence against the assumption of a universal, language-agnostic harm manifold within the specific low-resource languages studied. Warning: This paper contains example data that may be offensive or harmful.
Abigail Oppong, P SAM SAHIL, Tadesse Destaw Belay et al.· 0 citations
Relation extraction (RE) for low-resource languages is typically constrained by the lack of annotated corpora. We investigate the feasibility of cross-lingual RE for Romanian by combining automatic dataset translation with large language model (LLM) inference. We translate the SemEval-2010 Task 8 benchmark from English to Romanian using an LLM-based translation pipeline and evaluate Gemma 4 31B under zero-shot, few-shot, and QLoRA fine-tuned configurations, against four encoder baselines spanning 125M to 560M parameters: XLM- RoBERTa (base and large), Romanian BERT, and RoBERT- large. We assess two task formulations: relation classification with marked entities and end-to-end extraction. Our results show that Romanian incurs a 3 to 5 percentage point (pp) drop relative to English in prompt-only settings, that few-shot prompting provides marginal gains over zero-shot, and that QLoRA fine-tuning improves macro F1-Score by more than 22 percentage points in both languages while reducing the cross-lingual gap from 3.3 to 1.4pp. The encoder baselines come within 1-4pp of QLoRA Gemma on Romanian despite being 50-250 times smaller, with monolingual Romanian BERT at 125M parameters matching multilingual XLM-R at 278M. The case for using a 31B model for single-task RE on Romanian is therefore weak in deployment scenarios where compute matters. We release the translated dataset, evaluation code, and trained models.
With recent advancements, Small language models (SLMs) are increasingly used as preprocessors to handle query classification, routing, and candidate selection in retrieval pipelines, but they are nearly always prompted in English, even when users search in Hindi, Bengali, or code-mixed forms. We test whether prompting the same (frozen) SLM in three typologically diverse languages and aggregating the outputs can improve classification without retraining or translation. Nine decoder-only models (1B--9B parameters) evaluated on four public benchmarks show that confidence-weighted fusion of English, Hindi, and Bengali predictions raises macro-F1 by 3--5 points over English-only baselines, with the strongest gains on binary and coarse intent tasks. Parallel execution keeps latency within 1.2--1.4× of the single-language baseline. A paraphrase-only ensemble under identical conditions reaches only +1.4~F1 on average, suggesting that cross-lingual diversity rather than surface-level input variation drives the gain. Because no additional data, training, or translation services are required, our method may be useful when scaling to larger models is out of reach.
A new method for surgically removing training examples from a model reveals that as datasets grow, the link between what a model learns and what it produces dissolves.
