Skip to content

Spatial and Semantic Reasoning for LLM-Driven Robot Navigation via MCP

Sep 2026 · arXiv.org · Vol abs/2609.27340 · 0 citations · 17 references
Computer Science

TL;DR

This work proposes a non-invasive framework that connects LLM reasoning with ROS-based navigation through a navigation-oriented representation layer, exposed through the Model Context Protocol as standardized, reusable tools so that any MCP-compatible LLM can access them without robot-specific wrappers.

Abstract

Large language models (LLMs) are increasingly used as natural-language interfaces for robotic systems, yet their integration with Robot Operating System (ROS)-based navigation remains limited by two gaps. First, navigation data such as occupancy grids are represented as raw geometric messages that are difficult for LLMs to use directly as spatial or semantic context. Second, adding LLM-driven capabilities often requires custom wrappers or robot-specific interfaces, limiting reuse across systems. To address these challenges, we propose a non-invasive framework that connects LLM reasoning with ROS-based navigation through a navigation-oriented representation layer, exposed through the Model Context Protocol (MCP) as standardized, reusable tools so that any MCP-compatible LLM can access them without robot-specific wrappers. The visual map modules transform occupancy grids into metric, pose-aware images for goal reasoning, while the semantic annotation modules record waypoint-level observations with robot poses. We evaluate the framework on three tasks: autonomous mapping, spatial reasoning-based navigation, and semantic reasoning-based navigation. The results show that the evaluated LLM backends use these representations to achieve over 97% map coverage and select spatial or semantic navigation targets from natural-language instructions in a simulated indoor environment. This demonstrates representation-mediated LLM navigation without modifying the existing ROS navigation stack.

View source

Similar papers

Preprint Oct 2026

OntoPlan: An Ontology-Grounded Scene Representation and Agentic Framework for Scalable Robot Task Planning

Large language model (LLM)-based robot task planning is promising for open-ended instruction following, but degrades on long-horizon tasks in large environments. When spatial information is conveyed to the LLM through text, the model can fail to capture spatial context, and token cost grows with environment size. Gener...

Hyeongwoo Nam, Woongje Cho, Juwon Kim et al. · 0 citations
Open access Aug 2026

LLM-Based Semantic Navigation on a Low-Cost ROS Mobile Robot: A Hybrid Edge–Cloud Architecture

A semantic-aware autonomous navigation framework implemented on a ROS 1 (Melodic) mobile robot equipped with a two-dimensional light detection and ranging (LiDAR) sensor and an RGB-D camera that offers a low-cost, extensible basis for language-guided robots in smart environments.

M. Paun, F. Dragomir, Otilia Elena Dragomir et al. · 0 citations
2026

CoIN: Interactive Navigation With Counterfactual Reasoning via Vision–Language Models

Interactive navigation requires robots to actively modify cluttered environments to create traversable paths, going beyond passive obstacle avoidance. However, existing methods either depend on global maps and lack the reasoning capabilities to make interaction decisions from local observations, or are restricted to in...

Kang-Jie Zhou, Zhe-Jia Wen, Zhi-Yong Zhuo et al. · 0 citations
Sep 2026

A Topological Representation with Object-Path Graphs for Open-Vocabulary Instance Navigation

This work proposes an object--path graph that unifies open-vocabulary semantic reasoning with topological navigation, and introduces a navigation strategy that combines global path planning with local inter-node execution through lightweight node localization and semantic visual servoing, enabling navigation directly o...

Lin-Wei Zheng, Dao-Jie Peng, Bing-Tao Wang et al. · 0 citations
Review Open access Aug 2026

Vision-and-Language Navigation: A Component-Centric Survey of Interactions, Coupling, and Deployment

This survey revisits VLN from a component-internal perspective, viewing it as a navigation system composed of internal components such as instructions, environment representations, and embodied agents, and organizing existing work around the functions and interactions of these components.

Xiang-Xun Wu, Yin-Sheng Wu, Xiaojiang Peng · 0 citations

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.