Aug 2026· Conference on Multimedia Information Processing and Retrieval· pp. 28-34· 0 citations· 31 references
Abstract
Spatial understanding has been a challenging task for existing Multi-modal Large Language Models (MLLMs). Previous methods leverage large-scale MLLM finetuning to enhance MLLM’s spatial understanding ability. In this paper, we present a data-efficient approach. We propose a LLM agent system with strong and advanced spatial reasoning ability, which can be used to solve the challenging spatial question answering task in complex indoor warehouse scenarios. Our system integrates multiple tools that allow the LLM agent to conduct spatial reasoning and API tools interaction to answer the given complicated spatial question. Extensive evaluations demonstrate that our system achieves high accuracy and efficiency in tasks such as object retrieval, counting, and distance estimation. On the 2025 AI City Challenge Physical AI Spatial Intelligence Warehouse dataset, our system achieves 95.86% accuracy, ranking 1st place among all teams. On the SpatialBench benchmark, our system further attains 84.2% accuracy, setting a new state-of-the-art for zero-shot spatial question answering. The code is available at https://github.com/hsiangwei0903/SpatialAgent.
Large Language Models (LLMs) can be applied to various real-world tasks, many of which require spatial reasoning, like pattern-based spatial search, Point of Interest (POI) recommendation, and itinerary or travel planning. However, on their own LLMs have been shown to lack reliable spatial reasoning capabilities, espec...
Nicole R. Schneider, Nandini Ramachandran, Kent O'Sullivan et al.· ACM Transactions on Spatial...· 0 citations
Multimodal Large Language Models (MLLMs) have been widely applied in embodied agents and robotics for vision tasks. However, MLLMs suffer from limitations in spatial reasoning and perception. Post-training techniques have shown promise for improving these capabilities; yet, there are limited data-driven surveys of such...
Mahdieh Mallahnezhad, Masoud Jafaripour, Mohamadali Tofigh et al.· 0 citations
Comprehensive video understanding is crucial for advancing artificial intelligence toward the intricate dynamics of the physical world. While recent advances in Multimodal Large Language Models (MLLMs) have demonstrated remarkable capabilities in video understanding, existing benchmarks remain confined to simple scene-...
Seoyeon An, Hyeonseo Jang, Minsu Kim et al.· 0 citations
Existing spatial reasoning benchmarks mainly test spatial perception: reading off relations already visible in the input. Yet real-world spatial intelligence demands predictive spatial reasoning: constructing a scene from observations, anticipating how an intervention changes it, and reasoning about the unseen outcome....
Hong-Xing Li, Jin-Yue Su, Ding-Ming Li et al.· 0 citations
TTL-SR is proposed, a geometry-aware Test-Time Learning framework for quantitative Spatial Reasoning that leverages geometric consistency constraints and unlabeled test data to adapt models to target domains to boost spatial reasoning performance.
Ge-Ge Zhang, Shuai-Cheng Niu, Gang Dai et al.· 0 citations
SpatialMind is introduced, a metric-scale VLM for spatial reasoning and future prediction that substantially outperforms both general and spatially specialized models on the authors' benchmark while achieving competitive zero-shot performance on VSI-Bench, OSI-Bench, and VLM4D.
Fei-Ran Wang, Xiao-Qi Wang, Zi-Wei Li et al.· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.
