Optimising Retrieval for Linguistic Question-Answering in European Portuguese: A Benchmark on Ciberdúvidas Da Língua Portuguesa

From manual retrieval to intelligent question answering: A four-stage evaluation of multilingual access in South African open scholarly repositories

This article evaluates multilingual information access in three selected South African open scholarly repositories through a four-stage analytical heuristic: manual retrieval, keyword retrieval, multilingual semantic retrieval, and retrieval-augmented generation (RAG)-based question answering. The framework serves as an organizing device for comparing access mechanisms across conceptual, retrieval, and answer-generation layers. Using 4944 open scholarly records, 400 English queries, 400 machine-translated isiZulu queries, and 800 relevance judgments, the study compares BM25 keyword retrieval, multilingual-E5-large semantic retrieval, and a Qwen2.5-based RAG pipeline. The isiZulu component is interpreted as a translated-query benchmark rather than as evidence of authentic isiZulu user search behaviour. The results show that English retrieval is already strong under BM25, whereas translated isiZulu queries benefit more clearly from multilingual semantic retrieval, especially at Top-10. However, these gains remain bounded by the three-repository case-study corpus and do not remove cross-language access gaps. The RAG stage does not improve retrieval ranking beyond dense retrieval, but it changes the form of service by transforming retrieved evidence into natural-language answers. Answer evaluation indicates that larger generator scale improves faithfulness and evidence support, while completeness and isiZulu language quality remain limited. The findings suggest that multilingual semantic retrieval can partly reduce translated-query barriers in open repository access, but RAG-based library services require transparent evidence display, uncertainty signalling, and professional governance.

Beyond Multilingual Averages: MTEB-PT, a Benchmark for Portuguese Sentence Encoders

MTEB-PT is presented, a Portuguese benchmark constructed from a subset of MMTEB, comprising 14 existing datasets across Semantic Textual Similarity (STS), classification, retrieval, and reranking, and shows that language-specific fine-tuning still improves model performance in Portuguese, especially on task types that match the adaptation data most closely.

Semantic-Symbolic Knowledge Consensus for Multilingual Question Answering

Multilingual Question Answering (MQA) is primarily characterized by semantic-based and symbolic-based approaches, remain constrained by Language Confinement. Semantic methods, such as RAG and Agentic frameworks, suffer from linguistic bias that restricts retrieval and reasoning to the source language, while symbolic approaches struggle with cross-lingual schema alignment. We consider that integrating multilingual information from the semantic space into the symbolic space to formulate precise queries enables the effective utilization of global multilingual knowledge. In this paper, we propose SeSyCo, a Semantic-Symbolic Knowledge Consensus framework. Specifically, we leverage the semantic space to diverge monolingual queries into broad multilingual evidence, and subsequently utilize the symbolic space to eliminate language discrepancies, converging the gathered information into a robust consensus for precise SPARQL generation. Extensive experiments on the MLaKE dataset demonstrate that SeSyCo outperforms the strongest baseline by 10.7% in multi-hop settings, validating that establishing a robust multilingual consensus is essential for enhancing MQA performance. The collection is available at https://github.com/YuZhang9408/SeSyCo.

A Comparative Evaluation of Embeddings and LLMs in a Greek Book Publisher Setting - The CUP Dataset

We present CUP, a Greek book retrieval benchmark consisting of 868 catalog records and 104 expert-annotated queries with graded relevance judgments. We evaluate sparse (BM25), dense (sentence-transformers), hybrid, and LLM-assisted retrieval methods in this book-search setting. Multilingual embeddings outperform Greek-specific models, while hybrid retrieval performs best overall. A query-level analysis shows that BM25 excels at named-entity queries, while dense and hybrid methods improve natural-language, noisy, cross-lingual, and concept queries. Field-aware prompting has model-specific effects, while LLM TOC summarization improves TOC-only retrieval and LLM post-filtering improves early-stage retrieval at a high cost. Overall, CUP enables real-world evaluation of Greek retrieval across lexical, semantic, noisy, and cross-lingual queries.

DocuMind: An Offline Multilingual Retrieval-Augmented Generation System for Document Question Answering Using Local Large Language Models

ABSTRACT DocuMind is a fully offline, privacy-preserving, multilingual Document Question Answering system built on the Retrieval-Augmented Generation (RAG) architecture. The system enables users to upload documents in any language and ask questions in their native language — including Kannada, Hindi, Tamil, Telugu, and English — receiving accurate, contextually grounded answers from a locally running Large Language Model (LLM) with zero cloud dependency and zero recurring cost. The system employs Language-Agnostic BERT Sentence Embeddings (LaBSE), a multilingual model supporting 109 languages, to map all text into a shared 768-dimensional semantic vector space, enabling true cross-lingual retrieval without any translation step. Document embeddings are persisted in a local ChromaDB vector database and answer generation is performed by LLaMA 3.2 (3B parameters) running locally via the Ollama runtime. Evaluation using RAGAS-style metrics across five languages demonstrates Faithfulness scores of 0.77 to 0.87 and average query latency under 8 seconds on a standard consumer laptop with 8 GB RAM. KEYWORDS Retrieval-Augmented Generation, Large Language Models, Multilingual NLP, LaBSE, ChromaDB, Ollama, LLaMA, Document Question Answering, Indic Languages, Offline AI, Natural Language Processing, Cross-lingual Retrieval.

Cross-Lingual Information Retrieval for Indonesian–Javanese Documents Using LLM-Based Reranking