Skip to content
Log in

Similar papers

#small language model Open access Aug 2026

Retrieval Granularity as Evidence Design in Small-Model RAG Question Answering: A Diagnostic HotpotQA Study

Results align with a diagnostic perspective on chunking: using evidence at a task-appropriate level of granularity can improve grounding, auditability, and answer quality, but the observed patterns should be interpreted within the HotpotQA distractor setting, fixed generator, and tested context budgets.

Weimao Ke, Lixia Yang, Mengyang Xu · 0 citations
Preprint Jul 2026

Testing Retrieval-Augmented Generation Systems with Chunk Coverage

Retrieval-Augmented Generation (RAG)-based systems\footnote{For brevity, RAG-based systems are referred to as RAG systems throughout this paper.} are increasingly deployed in high-stakes settings where correct behaviour depends not only on the language model but also on the retrieval component that selects external documents at inference time. While existing RAG evaluation metrics assess retrieval and generation quality on a per-query basis, typically relying on query-level test oracles such as reference answers or relevance annotations, they provide limited insight into whether a test suite adequately exercises the retrieval behaviour of the system as a whole. In this paper, we introduce Chunk Coverage (CC), an oracle-independent test adequacy criterion for testing the retrieval component of RAG systems. CC measures the fraction of corpus chunks that are retrieved at least once across a test suite, providing a structural view of which parts of the retrieval space have been exercised. We further show how CC can be used to guide test selection and generation by prioritising queries that expand coverage of previously unexercised retrieval regions. We evaluate CC on clinical and financial RAG system scenarios. CC-guided testing reaches 50% of attainable coverage 1.7x faster than random selection and 4.2x faster than redundancy-biased strategies. Moreover, CC improves fault detection effectiveness (APFD) by 10% to 25% over random, indicating earlier discovery of distinct retrieval faults. These results show that CC captures retrieval diversity relevant to effective testing without requiring test oracles.

Jinhan Kim, Samuele Pasini, Paolo Tonella · 1 citation
Preprint Aug 2026

Search-GRT: Guided Retrieval Training of Search Agents to Optimize for Complex Question Answering

The effective use of search engines by large language models (LLMs) remains a significant challenge, particularly in complex, multi-hop question-answering (MHQA) tasks. These tasks require the model to decompose questions into subqueries, retrieve relevant information, and synthesize answers from multiple sources, often leading to cascading errors due to poor retrieval in early stages. Reinforcement learning (RL) has shown promise in improving LLMs'search capabilities, but it often suffers from sparse rewards during training, hindering the model's ability to learn effectively. To address these challenges, we introduce Guided Retrieval Training (GRT), a novel method that improves the performance of a search agent by restricting the retrieval process during RL training using ground truth information. By focusing on a curated set of relevant documents, GRT provides the model with a stronger learning signal, mitigating the problem of sparse rewards and improving its ability to generate accurate subqueries and synthesize correct answers. Our experimental results demonstrate that GRT achieves consistent performance improvements over existing methods, such as Search-R1, across a wide range of question-answering (QA) tasks. Notably, GRT excels in MHQA tasks, achieving over 40% improvements in performance. Additionally, GRT enhances training efficiency by achieving better QA performance with fewer training steps.

Aounon Kumar, Sudipta Paul, Vivek Kulkarni et al. · 0 citations
Open access Aug 2026

An Optimization Framework for Retrieval Augmented Generation in Indonesian Educational Question Answering

The Retrieval-Augmented Generation (RAG) approach has been widely adopted to produce responses that are more closely aligned with a predefined knowledge context. However, many RAG implementations have not undergone systematic optimization of their retrieval and generation components, resulting in outputs that do not always correspond accurately to the reference context. This study developed a RAG optimization framework for Indonesian-language educational question answering using a Human-Computer Interaction learning corpus as a case study. In the retrieval stage, the study evaluated chunking strategies, multilingual embedding models, and the use of a reranker. Evaluation was conducted using Mean Reciprocal Rank (MRR), Normalized Discounted Cumulative Gain (nDCG@K), and Hit@K. In the generation stage, candidate Large Language Models (LLMs) were assessed using RAGAS metrics, namely Context Precision (CP), Context Recall (CR), Faithfulness (F), Answer Relevancy (AR), and Answer Correctness (AC). Experimental results showed that the GTE configuration with fixed-size chunking and a reranker yielded the best retrieval performance, achieving an MRR of 0.9082, nDCG@5 of 0.9215, and Hit@5 of 0.9655. In the generation stage, Gemma 4 E4B exhibited the most balanced answer quality. The resulting framework provides a procedure for selecting retrieval and generation settings for a given corpus.

I. K. R. Arthana, N. Gunantara, Made Sudarma et al. · 0 citations
Book Open access Jul 2026

SCORE-RAG: Self-Correcting Exploration-Exploitation Retrieval for Multi-hop Question Answering

Retrieval-augmented generation (RAG) has emerged as a promising paradigm to enhance Large Language Models (LLMs) with external knowledge, effectively mitigating hallucinations and broadening the model's knowledge coverage. Despite recent advances, existing RAG methods fundamentally assume static query understanding, where the query is interpreted once before retrieval. This assumption proves inadequate for multi-hop questions, where comprehending the query itself often requires retrieval support, creating a chicken-and-egg dilemma between query understanding and information retrieval. To address this challenge, we propose SCORE-RAG Self-COrrecting Exploration-Exploitation REtrieval, a novel framework inspired by the explore-exploit paradigm in decision theory. SCORE-RAG reformulates multi-hop RAG as a two-phase adaptive process: exploration for dynamic query understanding, followed by exploitation for precise evidence gathering. Specifically, SCORE-RAG first performs exploratory retrieval with multi-perspective queries to resolve ambiguities and discover key entities and relations, then conducts targeted exploitation retrieval guided by the refined understanding to construct coherent evidence chains, and finally applies self-correction mechanisms to verify consistency and repair potential errors. Through this integrated approach, SCORE-RAG enables adaptive query comprehension, reduces error accumulation via self-verification, and produces interpretable reasoning chains for accurate answer generation. Extensive experiments on HotPotQA and 2WikiMultihopQA demonstrate that SCORE-RAG significantly outperforms existing state-of-the-art RAG frameworks, achieving substantial improvements particularly on complex multi-hop questions requiring deep reasoning.

Shuran Zhou, Rui Ling, Junan Chen et al. · 0 citations
Jul 2026

RAGnRoll: Learning to Iteratively Retrieve and Generate Attributable Answer Snippets

The rapid adoption of generative search engines has marked a significant shift in information retrieval. New approaches leverage Large Language Models (LLMs) to provide synthesized, contextually rich responses in natural language to directly satisfy users’ information needs. Due to their generative nature, these systems benefit from enhanced reliability by integrating explicit references to supporting sources in their outputs, otherwise known as Attribution. Teaching LLMs this capability is a challenging task that heavily depends on training data quality and strong training signals, especially for scenarios that require digesting long contexts and providing comprehensive answers. Planning and decomposition have notoriously helped language models produce better answers, but their impact on attribution remains relatively unexplored. This work proposes RAGnRoll, a language model for attributed answer generation within a multi-round Retrieval-Augmented Generation (RAG) framework. We leverage the iterative nature of multi-round RAG to train an LLM to incrementally build answers guided by subqueries. RAGnRoll is trained using a causal language modeling objective and a small, specially crafted dataset. Empirical evaluations on four different datasets for attributed long answer generation (ASQA, ELI5) and information-seeking (HAGRID and CORAL) demonstrate RAGnRoll's impact on both answer correctness and attribution accuracy. We extensively evaluate RAGnRoll training approach against baseline methods and benchmark its performance at inference against strong state-of-the-art models. RAGnRoll achieves competitive performance with advanced approaches without relying on extensive data or complex training.

Hanane Djeddal, Laure Soulier, K. Pinel-Sauvagnat et al. · 0 citations