AVA-VLM is proposed, an Adaptive Visual Attention-Vision Language Model that follows a human-inspired coarse-to-fine strategy: it first reasons over a low-resolution global image and requests a high-resolution local crop only when detailed inspection is needed.
Abstract
Existing construction-site Vision-Language Model (VLM) studies have primarily adapted pretrained VLMs through direct QA-style fine-tuning from a single global image, but we argue that this paradigm remains limited in operational range, reliability under reduced-resolution inputs, and inference efficiency. To address these limitations, we propose AVA-VLM, an Adaptive Visual Attention-Vision Language Model that follows a human-inspired coarse-to-fine strategy: it first reasons over a low-resolution global image and requests a high-resolution local crop only when detailed inspection is needed. We further introduce a region-aware Chain-of-Thought dataset that teaches when to inspect, where to crop, and how to use local evidence. Experiments show that, for violation identification, AVA-VLM improves overall F1 from 62.0 to 75.1 while using only 30.6% of the baseline visual-token budget; for long-distance PPE-violation cases, F1 improves from 16.0 to 63.6. These results demonstrate AVA-VLM's improved robustness to distant and reduced-resolution visual evidence with substantially lower visual-token usage.
Generative vision-language models (VLMs) such as Qwen-VL and LLaVA achieve strong zero-shot performance on tasks overlapping with their pretraining distribution, yet fail on specialized domains where the required discriminative features were never learned, a regime we term distant out-of-distribution (OOD). Standard ad...
Hung-Jen Chen, Yuek F. Ho, Ting-Yao Huang et al.· European Conference on Compu...· 0 citations
It is suggested that human gaze provides an effective and practical guidance signal for improving the computational efficiency of Vision-Language Models without requiring changes to the underlying models, offering a simple and general approach for reducing visual inference cost while largely preserving answer quality.
VIVAS is proposed, a framework built upon the unified token space paradigm, which introduces a dense-structural-semantic vision tokenizer, which expands the textual vocabulary into a unified vision-language vocabulary by incorporating a visual vocabulary.
Zhe-Han Kan, Yu-Bo Zhu, Xing-Hua Jiang et al.· 1 citation
This work proposes a principled VLM TTA method called \algname, and theoretically reveals that the InfoNCE loss can be neatly reformulated as a Wasserstein OT formulation, thereby unifying the objectives of the inference and adaptation of VLMs to achieve their mutual benefits.
Qi Yu, Zhichen Zeng, Katherine Tieu et al.· 0 citations
Vision-language-action (VLA) models encode semantic information from vision-language pretraining, but manipulation also requires precise spatial reasoning. We present GeoBridge-VLA, a two-stage method for learning geometric features from a pretrained VLA's frozen visual encoder and using them for action prediction. Sta...
Hyunjung Song, Kangmin Kim, Loren Jinsoo Um et al.· 0 citations
OmniMoE-VL, a sparse VLM with a coupled visual-depth routed projector, enables question-dependent visual access while preserving the native visual-token sequence, and complements token-level expert routing in the vision and language stacks.
Long Qian, Bing-Ke Zhu, Jia-Qi Wei et al.· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.