Jul 2026· International Conference on the Theory of Information Retrieval· 0 citations· 48 references
Computer Science
TL;DR
ARF-RAG is proposed, an Adaptive Retriever-Friendly Retriever-Friendly Retrieval-Augmented Generation framework that adopts a role-unified mechanism, in which a single LLM simultaneously performs all retrieval-related and generation actions, including retrieval decision-making, query generation, and answer generation, enabling coherent optimization across all components.
Abstract
Retrieval-augmented Generation (RAG) has emerged as an effective paradigm for enhancing large language models (LLMs) with external knowledge, delivering substantial performance gains without costly parameter updates, particularly in low-resource settings. However, existing RAG systems still face several open challenges. Many approaches rely on static or heuristic retrieval strategies, limiting their ability to adapt retrieval strategies to different tasks with different reasoning requirements. Some methods build RAG systems based on independently optimized components, making joint optimization across all components infeasible. Moreover, empirical evidence shows that stronger retrievers do not necessarily lead to better end-to-end RAG performance, revealing a misalignment between retrieval quality and generation effectiveness and leaving considerable room for improvement in how retrieval is integrated into the RAG pipeline. Motivated by these challenges, we propose ARF-RAG, an Adaptive Retriever-Friendly Retrieval-Augmented Generation framework. ARF-RAG dynamically determines when to retrieve and further generates retriever-friendly queries that faithfully reflect the generator's current information needs while being optimized for the behavior of the given retriever. ARF-RAG adopts a role-unified mechanism, in which a single LLM simultaneously performs all retrieval-related and generation actions, including retrieval decision-making, query generation, and answer generation, enabling coherent optimization across all components. To effectively align the retriever and generator, we formulate RAG as a unified reinforcement learning (RL) problem. We design comprehensive reward functions that jointly account for answer correctness, retrieval effectiveness, and retrieval cost, enabling fine-grained policy optimization via proximal policy optimization (PPO). Extensive experiments. The codebase for reproducing the proposed framework is publicly available at: https://github.com/ii-research/ARF-RAG on three question-answering benchmarks demonstrate that ARF-RAG consistently outperforms strong non-RL and RL-based baselines on both simple and complex QA tasks. Further analyses show that ARF-RAG adaptively learns when to retrieve based on question complexity and generates more effective, retriever-aligned queries that resolve what to retrieve, highlighting the advantages of unified RL optimization for building robust and efficient RAG systems.
Retrieval-augmented generation (RAG) enhances large language models (LLMs) by incorporating external information, but traditional fixed retrieval processes struggle to adapt to complex task requirements. In recent years, reinforcement learning (RL) has been increasingly applied to train LLMs to autonomously invoke search tools, driving RAG to evolve from the passive information acquisition of a fixed pipeline to a trustworthy retrieval system with autonomous decision-making capabilities. This paper reviews the representative studies on the combination of LLMs, RAG and RL in recent years. It focuses on analyzing the role of RL in dynamic retrieval, process rewards, query optimization, etc., and compares the connections and evolutionary relationships among different methods. The research findings show that RL has gradually expanded from simply improving the accuracy of the final answer to optimizing queries, multi-round search, process decision-making and trustworthy screening, providing new ideas for enhancing the active retrieval ability of RAG and improving the credibility of information.
Zun-Long Hong· Applied and Computational En...· 0 citations
Retrieval-Augmented Generation (RAG) techniques have emerged as a promising direction to merge the non-parametric knowledge into Large Language Models (LLMs), thereby alleviating factual errors, hallucinations and outdated knowledge. Existing RAG methods, which append multiple retrieved documents or passages to the input of LLMs, will inevitably increase the context length, resulting in not only significant computational overhead and inference latency, but also performance degradation. Although reranking or compression modules have been introduced to address these challenges, they overlook the contextual preferences of the generative LLMs itself and may inadvertently discard information that is crucial for generation accuracy. To this end, we introduce InnerRAG, which incentivizes RAG via Inner Adaptive Context Selection. InnerRAG is a novel paradigm that empowers LLMs to autonomously select relevant context during generation. Our proposed InnerRAG endows the model to accurately identify the documents that are most helpful for generation from long contexts. By endowing the model with this capability, InnerRAG facilitates more effective exploitation of external knowledge without being misled by disturbed information, leading to substantial improvements in generation quality while maintaining computational efficiency. Extensive experiments across multiple benchmarks and human evaluations demonstrate that our method consistently outperforms state-of-the-art RAG baselines. Moreover, our framework is orthogonal and complementary to in-context RAG approaches, offering further performance improvements when combined.
Chenxu Cui, Lin Shen, Haihui Fan et al.· Annual International ACM SIG...· 0 citations
Multi-turn retrieval-augmented generation (RAG) improves question answering by decomposing evidence seeking into iterative retrieval and reasoning steps. Existing multi-turn RAG methods usually optimize when and how to retrieve while fixing the number of retrieved documents per step. However, we discovered that this fixed-TopK design is suboptimal: single-hop questions tend to benefit from fewer retrieval rounds with larger per-round evidence sets, whereas multi-hop questions require more retrieval rounds with smaller evidence sets to support stepwise reasoning. To bridge this gap, we introduce AdaRAG, a budget-aware adaptive RAG framework that learns how to retrieve under a hard document budget, including how many retrieval rounds to perform, how many documents to retrieve in each round, and which retrieval source to use. AdaRAG implements this idea with a two-level policy architecture. ModeHead, a lightweight retrieval-mode classifier, selects passage retrieval, graph retrieval, or answer generation; TopkHead, a budget-aware document-allocation classifier, selects a legal TopK after query generation according to the remaining budget. These discrete policy heads are decoupled from language-model token generation, enabling direct reinforcement-learning optimization through hierarchical GRPO after supervised action-format learning. Our experiments across five QA benchmarks demonstrate AdaRAG's good generalization performance under constrained document budgets. In detailed comparisons on HotpotQA, it surpasses the strongest baselines by an average of 10.8 percentage points in Exact Match (EM) and F1 score.
Jia-Nan Sun, Miao Zhang, Chen Chen et al.· Fall Joint Computer Conferen...· 0 citations
Retrieval-Augmented Generation (RAG) has become a fundamental paradigm for enhancing Large Language Models (LLMs) with external knowledge. However, while recent structure-augmented approaches organize documents into graphs to improve information access, their retrieval strategies remain largely static, relying on similarity ranking or static probability diffusion. We identify that this paradigm suffers from two inherent limitations in complex reasoning: popularity bias, where retrieval paths are trapped by high-degree distractors, and signal decay, where relevance signals attenuate over long reasoning chains. To overcome these challenges, we propose NaviRAG, a novel framework that reformulates retrieval as a reinforcement learning-driven dynamic navigation problem on schema-less knowledge graphs (KGs). Unlike passive diffusion, NaviRAG employs an agent that actively traverses the graph to act as a search-space pruning engine, identifying logical multi-hop reasoning paths. Technically, we introduce three key components: (1) Structure-Aware Query Expansion, which bridges the modality gap between unstructured queries and structured graph seeds for precise initialization; (2) Target-Driven Reward Shaping, which provides dense supervision based on semantic progress toward gold documents, effectively mitigating the sparse reward problem in large-scale graph traversal; and (3) a Multi-View Hybrid Reranking strategy that operates on the highly-pruned candidate subgraph, integrating policy confidence, semantic relevance, and global structural importance to ensure robust candidate selection. Extensive experiments on three multi-hop QA datasets and two single-hop QA datasets demonstrate that NaviRAG significantly outperforms baselines, achieving state-of-the-art performance in multi-hop QA while maintaining robustness in single-hop QA. Our code and data are available at https://github.com/CkingEW/NaviRAG.
Jinghong Lei, Wang Kun, Zhigang Chen et al.· Proceedings of the 32nd ACM...· 0 citations
Experimental results show that SAC-RAG reduces token consumption by 38%–58% at the cost of only a 1–2 percentage point EM drop, with EM actually improving after compression for reasoning-type questions, achieving the optimal quality–efficiency trade-off in terms of token consumption.
Deyu Zhang, Hongqiang Yu, Jinze Huo et al.· IEEE Access· 0 citations
DARE-RAG is proposed, an adaptive retrieval framework that activates LLM-based query expansion only for retrieval-challenging queries, and formulates expansion activation as a lightweight binary classification problem using probe retrieval signals, including score margin, variance, entropy, query length, and lexical specificity.
Li-Xiang Zhu· Poster Volume 0007 The 2026...· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.
