Forward-Looking Guidance for Iterative RAG in Multi-Hop Question Answering
Abstract
Multi-hop question answering aims to answer complex queries by identifying and integrating evidence distributed across multiple documents. Existing iterative RAG approaches address this task by decomposing complex questions into plans or sub-questions, followed by repeated retrieval, evidence extraction, and reasoning to accumulate the information needed for the final answer. Nevertheless, intermediate outputs produced during this iterative process are often optimized locally rather than guided by future reasoning needs, making them useful for the current iteration but suboptimal for the overall multi-hop reasoning process. We empirically investigate the potential of forward-looking guidance in iterative RAG, where the remaining planned reasoning trajectory is incorporated into each iteration. We analyze its effect across retrieval, evidence extraction, and reasoning, and find that such guidance is especially beneficial for evidence extraction and reasoning, leading to more effective intermediate outputs and consistent improvements over iterative RAG baselines.