Cross-Lingual Information Retrieval for Indonesian–Javanese Documents Using LLM-Based Reranking

Abstract

Cross-lingual information retrieval (CLIR) for low-resource regional languages remains challenging due to limited annotated training data, morphological complexity, and linguistic diversity. This paper systematically investigates Indonesian–Javanese CLIR by integrating traditional sparse lexical retrieval, multilingual dense retrieval, and advanced large language model (LLM)-based reranking within a multi-stage retrieval framework. To evaluate our approach, comprehensive experiments are conducted on the Indonesian–Javanese subset of the CLIRMatrix benchmark, comprising 20,000 Javanese documents, 1,000 Indonesian queries, and 11,000 graded relevance judgments. We compare BM25 with query and document translation, multilingual bi-encoder retrieval, and various LLM-based pointwise reranking methods. The experimental results show that BM25 combined with document translation provides the strongest initial baseline, achieving a Mean Average Precision (MAP) of 0.45, Mean Reciprocal Rank (MRR) of 0.71, and nDCG@10 of 0.55. While LLM-based reranking consistently improves weaker dense retrieval baselines, it often degrades search performance when applied directly on top of strong BM25-based retrieval. To address this, Reciprocal Rank Fusion (RRF) is introduced, which mitigates this degradation issue and achieves the best overall performance, reaching a MAP of 0.47, MRR of 0.74, and nDCG@10 of 0.57. These empirical findings highlight the critical importance of robust initial retrieval baselines and demonstrate that LLM-based reranking is most effective when combined with rank fusion strategies in low-resource CLIR settings.