Skip to content
Preprint

BEHAVIOR-Spatial Benchmark: Evaluating Spatial Understanding for Robotics

Oct 2026 · 2 citations · ⚡ 1 influential · 34 references
Computer Science

TL;DR

BEHAVIOR-Spatial is presented, a benchmark built on BEHAVIOR-1K that measures spatial question answering, point-based target grounding, and robot action to examine how VLM spatial understanding relates to VLA behavior.

Abstract

Robot tasks require grounding spatial language in physical scenes, but task success alone cannot establish whether spatial instructions are followed. We present BEHAVIOR-Spatial, a benchmark built on BEHAVIOR-1K that measures spatial question answering, point-based target grounding, and robot action to examine how VLM spatial understanding relates to VLA behavior. It comprises 3,256 spatial questions across 49 activities, 1,240 grounding queries across 14 manipulation tasks, and action evaluations covering those 14 tasks and 10 navigation tasks. Our manipulation protocol evaluates held-out configurations and instruction following when the requested target changes within a fixed training layout. The baseline achieves 41.6% on held-out configurations but only 16.1% under changed instructions. VLM adaptation and robot-data pretraining provide limited improvements, whereas oracle visual target cues raise these rates to 60.3% and 38.0%, respectively. Perception evaluations reveal policy-dependent correlations between VLM question-answering and grounding accuracy and VLA manipulation performance. Together, these findings motivate evaluating spatial instruction following alongside task completion and examining when spatial perception performance translates into robot behavior. Project website: https://arielchen07.github.io/behavior-spatial/.

View source

Similar papers

Preprint Oct 2026

Spa-Bench: A Real-Robot Benchmark for Spatial Grounding in Vision-Language-Action Models

Spa-Bench is introduced, a real-robot benchmark that evaluates spatial grounding across six tabletop tasks covering Object State, Relative Size, Referential Description, Relational Placement, Ordinal Position, and Counting, and shows that the evaluated VLAs can often execute the required manipulations but lack robust s...

Justin Tien-Smith, Matthew Pidden, Rolandos Alexandros Potamias · 1 citation
Sep 2026

RoboChrono: A Real Robot Benchmark for Streaming Task Understanding

Understanding ongoing robot manipulation requires models to interpret visual observations in relation to interaction history and task progress. We introduce RoboChrono, a benchmark for streaming task understanding comprising 39 scenarios and 34,713 evaluation instances, constructed from real robot executions and comple...

Yu-Zhou Wu, Long-Teng Fan, Zi-Meng Li et al. · 0 citations
#artificial intelligence Preprint Nov 2026

GeoMind: A Benchmark for Spatial Understanding in Robotic Manipulation

Robotic manipulation has advanced rapidly in mapping visual observations and language instructions to actions. However, reliable manipulation in open-world environments requires robust spatial understanding: the ability to identify targets in complex layouts (spatial reasoning) and adapt to scenes that continuously evo...

Jing-He Wang, Xin-Rui Cao, Duo Wu et al. · 0 citations
#machine learning Preprint Oct 2026

Encoded but Not in Control: Revealing the Grounding Gap in Vision-Language Robot Policies

Instruction following is central to language-conditioned robot policies: language should determine what to do when the same scene permits multiple valid actions. Yet successful execution alone cannot establish whether a policy follows the instruction or infers the task from the scene. We study this ambiguity through sc...

Shao-Han Jiang, Jia-Hang Cao, Qi-Duo He et al. · 1 citation
Preprint Nov 2026

STVLA: Spatial-aware and Temporal-memory forVision-Language-Action Models

Robotic manipulation often requires both spatial scene understanding and task history. However, many vision-language-action (VLA) policies rely primarily on current observations, without explicitly integrating spatial structure and task history. Appearance-based memories can be sensitive to occlusion and viewpoint chan...

Zhuo Han, Ye-Qiang Wang, Xin-Ran Wang et al. · 0 citations

VCN-Bench: A Video-Contextualized Navigation Benchmark for Spatial Reasoning over Prior Visual Experience

VCN-Bench is introduced, a VCN-Bench benchmark for probing closed-loop spatial reasoning over prior visual experience in MLLMs, and MV-DualVLN is proposed, a planning-oriented baseline that jointly leverages prior video and in-episode observations.

Si-Qi Zhang, Meng Wei, Chen-Yang Wan et al. · 0 citations

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.