Navigating to Interact: A Dual‐Graph Framework Orchestrating Semantic Exploration and Local Positioning for Household Assisting Robots

Abstract

Performing tasks for household assisting robots in complex, multi‐room environments requires embodied agents to accomplish long‐horizon cross‐room navigation and precise object interaction. However, existing approaches typically focus either on visual navigation without physical interaction or on manipulation tasks restricted to single‐room environments. To address these challenges, we propose a hierarchical framework that combines global semantic exploration with local interaction‐aware positioning. At the global level, a Dual‐Graph Navigation policy maintains a Dynamic Scene Topology Graph for online spatial memory and leverages a Prior Knowledge Graph to guide target‐oriented search through semantic object‐object correlations. At the local level, an Interaction‐Oriented Positioning module identifies interaction‐feasible docking poses on local occupancy maps for reliable manipulation. Extensive experiments on the large‐scale ProcTHOR‐10 k benchmark demonstrate that our framework achieves robust performance in navigation efficiency and task success rate, verifying its effectiveness in successfully accomplishing complex household tasks in diverse, procedurally generated environments. © 2026 Institute of Electrical Engineers of Japan and Wiley Periodicals LLC.