Translate, Search, or Answer: Cost-Aware Cross-Lingual Retrieval for Kazakh Small Language Models
TL;DR
This work systematically compares zero-shot parametric generation, in-language retrieval, and cross-lingual (translate-then-retrieve) web search using three 4B-parameter SLMs in both reasoning and non-reasoning modes and proposes a training-free, self-aware router that uses majority voting over repeated self-verification decisions to determine when to search the web, and when to escalate to a more capable cloud model.
Abstract
Small Language Models (SLMs) enable efficient deployment, but their limited parameter count constrains factual knowledge, particularly in low-resource languages like Kazakh. Integrating live web search can address this limitation, though its effectiveness is difficult to measure due to sparse in-language web indices and answer leakage during benchmarking. In this study, we systematically compare zero-shot parametric generation, in-language retrieval, and cross-lingual (translate-then-retrieve) web search using three 4B-parameter SLMs in both reasoning and non-reasoning modes. To evaluate factuality without search-engine leakage, we introduce machine-translated Kazakh versions of the FreshQA and DefAn benchmarks, and use GPQA as a Google-proof adversarial control. We also assess robustness across three prompt complexities, from simple JSON constraints to adversarial warnings that instruct the model to treat potentially unreliable context with caution. Finally, we propose a training-free, self-aware router that uses majority voting over repeated self-verification decisions to determine when to answer parametrically, when to search the web, and when to escalate to a more capable cloud model. Our results show that cross-lingual retrieval substantially outperforms in-language search on global factuality tasks, nearly doubling accuracy on FreshQA, while direct in-language search remains preferable for localized cultural queries. The choice of retrieval language depends on the task and does not always favor English. Additionally, cross-lingual retrieval is not consistently superior, because the best option depends on where relevant information is indexed. Pareto analysis indicates that cross-lingual search is on or near the optimal accuracy–latency frontier, adding minimal overhead compared to direct search. The router identifies which query types warrant retrieval, and as a system it tracks or exceeds always-search accuracy while issuing fewer searches and approaching the always-cloud ceiling at a fraction of its cost; per-query discrimination within a task family is weaker, which we quantify explicitly. Overall, this work offers a framework for optimizing and accurately measuring cross-lingual RAG pipelines in low-resource settings.