Oct 2026· IEEE Robotics and Automation Letters· Vol 11, pp. 11330-11337· 0 citations· 39 references
Abstract
Visual Language Navigation (VLN) enables robots to follow natural language instructions to navigate visually perceived environments. Typically, VLN systems are trained on multi-modal datasets that pair visual scenes with navigation instructions. While prior work has focused on generalising to unseen environments, linguistic variability in human instructions remains largely unmodeled. This work explores the sensitivity of state-of-the-art VLN models to natural instructions that fall outside the highly constrained subsets found in standard benchmarks. We conduct a user study where diverse participants provide spoken navigational instructions to multiple robot embodiments. We then propose a high-level taxonomy to characterise variability of navigational instructions grounded in Human Robot Interaction (HRI). We use this framework to show that instruction style is influenced by human traits and robot embodiment. Finally, we evaluate multiple VLN model families on the collected data and demonstrate that altering instruction style alone, while keeping environments and trajectories fixed, leads to substantial performance degradation across all architectures. These findings highlight linguistic variability as a critical and underexplored challenge for robust, human-centered VLN systems.
This work uses intervention-based metrics that measure how visual observations, instructions, and visual memory causally influence navigation decisions and shows that these navigation policies are sensitive to all input modalities and do not depend on a single one.
This survey revisits VLN from a component-internal perspective, viewing it as a navigation system composed of internal components such as instructions, environment representations, and embodied agents, and organizing existing work around the functions and interactions of these components.
HumanoidVLN, a physics-grounded simulator and benchmark for VLN across diverse humanoid embodiments, and compatibility with NaVILA, DualVLN, StreamVLN, and JanusVLN are presented.
Robot navigation in dynamic, human-centered environments requires socially-compliant decisions grounded in robust scene understanding. Recent Vision-Language Models (VLMs) exhibit promising capabilities such as object recognition, common-sense reasoning, and contextual understanding, capabilities that align with the nu...
Nathan Tsoi, M. Munje, T. Oberoi et al.· 0 citations
HINT (Human-INTent INcepTion), an agentic framework inspired by the human manipulation principles: semantic intent changes sparsely at manipulation-pattern transitions, whereas continuous control primarily depends on the evolving object-hand relationship.
Ming-Yu Mei, Haojie Xu, Shi-Hao Jin et al.· 0 citations
NaViRRator, a framework that translates user-specified start and goal locations on such maps into navigation instructions for a pretrained vision-and-language navigation (VLN) policy, is presented, supporting route-grounded language as a modular interface between human-readable maps and pretrained navigation policies.
Ayun Lee, Jiseon Kim, Giseop Kim· arXiv.org· 1 citation
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.