Jul 2026· The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences· 0 citations· 2 references
TL;DR
Key insights are provided for optimizing cross-lingual vision-language alignment of LVLMs and advancing practical multimodal applications in UAV scenarios and differences in language structures: East Asian languages depend on word order for semantic expression, whereas Western languages feature complex lexical morphology, significantly affect attention allocation and VG accuracy.
Abstract
Abstract. Visual Grounding (VG) is a core multimodal task that localizes image targets via natural language descriptions, and it is crucial for Unmanned Aerial Vehicle (UAV) applications. However, existing remote sensing (RS) VG datasets primarily rely on rule-driven explicit descriptions, which are inconsistent with real-world demands for interpreting implicit descriptions based on context, common sense, or domain knowledge. In addition, the cross-lingual robustness of Large Vision-Language Models (LVLMs) in implicit VG remains to be thoroughly investigated. This study evaluates the cross-lingual performance of Qwen2.5-VL-7B and InternVL3.5-8B across nine languages, incorporating analyses of text length dynamics, visual attention, and language structural effects. The results demonstrate that Qwen2.5-VL-7B exhibits outstanding performance in maintaining consistent task paradigm alignment (explicit VG outperforms implicit VG) and balanced text output, which benefits from the syntactic stability and low cognitive load of East Asian languages. In contrast, InternVL3.5-8B presents task paradigm misalignment, uncontrolled text expansion, and generative hallucinations. Furthermore, differences in language structures: East Asian languages depend on word order for semantic expression, whereas Western languages feature complex lexical morphology, significantly affect attention allocation and VG accuracy. This study provides key insights for optimizing cross-lingual vision-language alignment of LVLMs and advancing practical multimodal applications in UAV scenarios.
This work proposes GrabVG, a novel visual grounding framework inspired by human visual search that generates a compact set of reliable object hypotheses through distillation-guided proposal induction and text-aware hypothesis filtering, substantially reducing background distractions and semantic mismatches.
Chaowei Wang, Yan Di, Jingjun Sun et al.· 0 citations
Unmanned aerial vehicles (UAVs) are used for a wide range of tasks, many of which are autonomous. Since UAVs operate in three-dimensional space, visual data presents a rich source of input for them. Meanwhile, the rapid development of artificial intelligence has given rise to vision-language models (VLMs). These models not only recognize objects in a scene, as traditional computer vision techniques do, but also demonstrate a strong understanding of scene context, enabled by large language models (LLMs) serving as one of their core components. However, applications as specialized as aerial robotics typically require custom-trained models, which demand significant computational resources, large datasets, and expensive hardware. Motivated by these limitations, we investigate whether a state-of-the-art, general-purpose VLM can be leveraged for UAV-relevant tasks through carefully designed prompts, without additional training. We evaluate the VLM’s understanding of aerial scenes through the task of image captioning, using visual question answering that targets spatial information within the scene. Furthermore, we utilize VLMs to automatically generate metadata for aerial datasets, enabling more complex future tasks. Results show that VLMs can generate accurate metadata with high agreement with human-annotated data even without retraining. They also demonstrate that careful prompt engineering enhances the model’s ability to reason about and attend to spatial information, highlighting its potential in aerial robotics applications.
Elena Wachtler, Antonella Barisic Kulas, Tamara Petrović et al.· International Conference on...· 0 citations
MoRAL (Multimodal Reasoning for Autonomous Language Models), a two-stage fine-tuning pipeline that teaches Cosmos-Reason2-2B to first read a physics-encoded Bird's Eye View (BEV) representation and then reason over it for driving decisions, establishes a reproducible foundation for compact, physics-grounded VLM reasoning on mobile edge platforms.
Ambarish Govindarajulu Kaliamurthi, Kai Liu· 0 citations
Visual grounding is typically evaluated as a one-shot mapping from an informative referring expression to a visual target. This formulation misses a central property of real-world reference: initial referring expressions are often incomplete or ambiguous, requiring participants to establish shared understanding through interaction. We introduce a controlled evaluation framework for interactive visual grounding in large vision-language models (LVLMs), varying how much target information is provided upfront and how much must be acquired through dialogue. Across four human-grounded visual contexts and four interaction protocols, current LVLMs perform significantly below task-level human baselines. Interaction can help when follow-up questions refine or repair an initial target description. Performance is lowest when no initial description is provided and target information must be acquired through questions, indicating that proactive question-driven grounding remains difficult. LVLMs are also poorly calibrated, often reporting confidence that exceeds their empirical accuracy. Follow-up studies confirm these patterns across varied description sources (human versus AI), reasoning efforts, repeated interactions, description providers, and visual contexts. Overall, interactive visual grounding remains challenging, requiring visual matching, information seeking and synthesis.
Vision-language-action (VLA) models provide a scalable path toward generalist robotic manipulation by integrating visual perception, language understanding, and continuous action control. However, we reveal a critical limitation of VLA architectures: the action expert has limited access to the 3D geometric and 2D semantic information available in VLM features. This accessibility gap weakens perceptual grounding and limits performance on fine-grained robotic manipulation. To address this issue, we propose V-Link, which explicitly recovers visual representations during the vision-language (VL) to action (A) feature transfer. Specifically, V-Link learns complementary Spatial and Semantic Query representations within the VLM and injects them into Action DiT through asymmetric pathways. Semantic Queries complement the original VLM image tokens, whereas Spatial Queries provide dedicated geometric conditioning for spatially grounded action generation. Across LIBERO, LIBERO-Plus, and RoboTwin 2.0, our V-Link improves the average success rate over base model GR00T N1.6 by +1.9%, +31.2%, and +18.8%, respectively. On the AGIBOT A3 Ultra, V-Link further achieves gains of +20% and +24% on two real-world humanoid tasks.
Yehao Lu, Jiarui Yang, Yu-Ning Su et al.· 0 citations
Embodied navigation requires agents to translate heterogeneous goals and visual observations into actions across tasks, environments, and robot embodiments. Modern vision-language models (VLMs) already encode spatial priors for visual grounding, spatial reasoning, and pointing, but these capabilities are rarely elicited directly for robot control. Existing navigation systems instead rely on task- or embodiment-specific components, fragmenting perception, reasoning, and action while offering limited generalization. Here we present LightNav-0, a compact generalist embodied navigation model that elicits the spatial intelligence of a pretrained VLM and aligns it with navigation, without task-specific prediction heads. LightNav-0 represents diverse navigation tasks through a unified token interface: dual-channel pointing expresses task-, scene-, and embodiment-agnostic spatial intent, while a residual vector-quantized action tokenizer maps this intent to precise, embodiment-specific trajectories. Together with temporally aware visual history compression, ER mid-training, supervised fine-tuning, and reinforcement learning, this formulation supports instruction following, open-vocabulary object navigation, and visual tracking within a single model. The navigation training corpus spans 2K+ scenes and 4K+ hours of embodied navigation data. LightNav-ER, the embodied-reasoning checkpoint used to initialize LightNav-0, attains the highest complete-set average across 8 embodied-reasoning benchmarks, while LightNav-0 achieves state-of-the-art monocular success rates across all 10 public navigation simulation settings. Real-world evaluations further demonstrate zero-shot generalization across robot embodiments, diverse scenes, and static and dynamic targets. These results establish compact VLMs as a unified and transferable backbone for generalist embodied navigation.
Shao-An Wang, Ao-Cheng Luo, Fei Huang et al.· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.
