This work proposes a non-invasive framework that connects LLM reasoning with ROS-based navigation through a navigation-oriented representation layer, exposed through the Model Context Protocol as standardized, reusable tools so that any MCP-compatible LLM can access them without robot-specific wrappers.
Abstract
Large language models (LLMs) are increasingly used as natural-language interfaces for robotic systems, yet their integration with Robot Operating System (ROS)-based navigation remains limited by two gaps. First, navigation data such as occupancy grids are represented as raw geometric messages that are difficult for LLMs to use directly as spatial or semantic context. Second, adding LLM-driven capabilities often requires custom wrappers or robot-specific interfaces, limiting reuse across systems. To address these challenges, we propose a non-invasive framework that connects LLM reasoning with ROS-based navigation through a navigation-oriented representation layer, exposed through the Model Context Protocol (MCP) as standardized, reusable tools so that any MCP-compatible LLM can access them without robot-specific wrappers. The visual map modules transform occupancy grids into metric, pose-aware images for goal reasoning, while the semantic annotation modules record waypoint-level observations with robot poses. We evaluate the framework on three tasks: autonomous mapping, spatial reasoning-based navigation, and semantic reasoning-based navigation. The results show that the evaluated LLM backends use these representations to achieve over 97% map coverage and select spatial or semantic navigation targets from natural-language instructions in a simulated indoor environment. This demonstrates representation-mediated LLM navigation without modifying the existing ROS navigation stack.
AnchorVLN is introduced, an open-vocabulary VLN system built on a simple rule: the VLM proposes semantics; geometry decides metrics, which enforces the semantic-geometry boundary without modifying the downstream autonomy stack.
Long Giang Vu, Cheng-Kai Yao, Yu-Xin Liu et al.· 0 citations
Large language model (LLM)-based robot task planning is promising for open-ended instruction following, but degrades on long-horizon tasks in large environments. When spatial information is conveyed to the LLM through text, the model can fail to capture spatial context, and token cost grows with environment size. Gener...
Hyeongwoo Nam, Woongje Cho, Juwon Kim et al.· 0 citations
A semantic-aware autonomous navigation framework implemented on a ROS 1 (Melodic) mobile robot equipped with a two-dimensional light detection and ranging (LiDAR) sensor and an RGB-D camera that offers a low-cost, extensible basis for language-guided robots in smart environments.
M. Paun, F. Dragomir, Otilia Elena Dragomir et al.· Future Internet· 0 citations
Interactive navigation requires robots to actively modify cluttered environments to create traversable paths, going beyond passive obstacle avoidance. However, existing methods either depend on global maps and lack the reasoning capabilities to make interaction decisions from local observations, or are restricted to in...
Kang-Jie Zhou, Zhe-Jia Wen, Zhi-Yong Zhuo et al.· IEEE Transactions on Automat...· 0 citations
This work proposes an object--path graph that unifies open-vocabulary semantic reasoning with topological navigation, and introduces a navigation strategy that combines global path planning with local inter-node execution through lightweight node localization and semantic visual servoing, enabling navigation directly o...
Lin-Wei Zheng, Dao-Jie Peng, Bing-Tao Wang et al.· arXiv.org· 0 citations
This survey revisits VLN from a component-internal perspective, viewing it as a navigation system composed of internal components such as instructions, environment representations, and embodied agents, and organizing existing work around the functions and interactions of these components.