SCORE-RAG: Self-Correcting Exploration-Exploitation Retrieval for Multi-hop Question Answering

Abstract

Retrieval-augmented generation (RAG) has emerged as a promising paradigm to enhance Large Language Models (LLMs) with external knowledge, effectively mitigating hallucinations and broadening the model's knowledge coverage. Despite recent advances, existing RAG methods fundamentally assume static query understanding, where the query is interpreted once before retrieval. This assumption proves inadequate for multi-hop questions, where comprehending the query itself often requires retrieval support, creating a chicken-and-egg dilemma between query understanding and information retrieval. To address this challenge, we propose SCORE-RAG Self-COrrecting Exploration-Exploitation REtrieval, a novel framework inspired by the explore-exploit paradigm in decision theory. SCORE-RAG reformulates multi-hop RAG as a two-phase adaptive process: exploration for dynamic query understanding, followed by exploitation for precise evidence gathering. Specifically, SCORE-RAG first performs exploratory retrieval with multi-perspective queries to resolve ambiguities and discover key entities and relations, then conducts targeted exploitation retrieval guided by the refined understanding to construct coherent evidence chains, and finally applies self-correction mechanisms to verify consistency and repair potential errors. Through this integrated approach, SCORE-RAG enables adaptive query comprehension, reduces error accumulation via self-verification, and produces interpretable reasoning chains for accurate answer generation. Extensive experiments on HotPotQA and 2WikiMultihopQA demonstrate that SCORE-RAG significantly outperforms existing state-of-the-art RAG frameworks, achieving substantial improvements particularly on complex multi-hop questions requiring deep reasoning.