Skip to content
Book Open access

How Much Visual Evidence Does a Grounding Agent Need? Budget Profiles for Agentic Remote-Sensing Visual Grounding

Nov 2026 · Proceedings of the 1st International Workshop on Agentic Multimodal Intelligence: Models, Benchmarks, and Applications · pp. 79-87 · 0 citations · 3 references

TL;DR

The HBB inference profiles of AgentRSVG, whose controller maintains cross-scale hypotheses, directs retrieval and zoom toward unresolved query constraints, and uses a grounding verifier to stop or continue are studied to identify a strong operating point for budget-constrained agentic grounding.

Abstract

Remote-sensing visual grounding does not have a uniform inference cost. A whole-scene view preserves layout but can erase tiny referents, whereas tile retrieval and zoom recover detail at additional latency. This paper asks a practical question: how much visual evidence should a grounding agent acquire before returning a horizontal box? We study the HBB inference profiles of AgentRSVG, whose controller maintains cross-scale hypotheses, directs retrieval and zoom toward unresolved query constraints, and uses a grounding verifier to stop or continue. Low, Main, and High profiles share trained weights and validation-frozen stopping thresholds; they differ only in caps on actions, tiles, proposals, and zooms, while Single removes the iterative loop from the common visual backbone. On DIOR-RSVG, Single obtains 69.20 Pr@0.5 and 61.25 mIoU. Low raises these values to 85.80 and 77.15 at 2.6 seconds per query, recovering 83.2% of the Single-to-Main accuracy gain. Main reaches 89.15 Pr@0.5 and 80.28 mIoU at 4.0 seconds, whereas High adds only 0.57 Pr@0.5 while increasing latency by 40%. On RSVG-HR, Main obtains 92.18 Pr@0.5 and 80.45 mIoU. Small-target diagnostics identify where extra computation is most useful, while verified positive cases illustrate successful localization across scale, repetition, geometry, and context. These results turn a single headline score into an explicit accuracy–cost frontier and identify a strong operating point for budget-constrained agentic grounding.

Read PDF

Similar papers

Preprint Oct 2026

TerraAgent: Demonstration-Grounded Multi-Agent Reasoning for Long-Context Remote Sensing Visual Question Answering

AVA, a training-free framework that wraps a frozen RS MLLM with three modules that improves average accuracy by up to 2.40 points over the strongest same-setting baselines, and raises the visual-dependency score from 0.28 to 0.49, is presented.

Jing-Wen Huang, Yi-Ming Fang, Han-Xiao Wu et al. · 0 citations
#machine learning Preprint Oct 2026

PROVE: Probabilistic Visual Grounding via Patch-Level Evidence for Vision-Language Models

Visual grounding with vision-language models such as CLIP typically produces a single deterministic relevance map for an image-query pair, without indicating whether the localisation is reliable or whether the referent is present at all. We introduce \textbf{PROVE} (\textbf{Pro}babilistic \textbf{V}isual Grounding via...

Mayank Nautiyal, Tian-Rui Zhang, Lin Ju et al. · 0 citations
Sep 2026

GroundingPI: A Grounding Foundation Model towards Physical Intelligence with Visual Primitives

Precise grounding matters. It specifies which object is the target and where that object is, even in clutter and for tiny objects, and it has to be fast enough for closed-loop control. Yet vision-language-action (VLA) and world-action models (WAMs) take perception from general-purpose vision-language and video-generati...

Qi-Ze Yu, Lian-Rui Fan, Bo-Yu Chen et al. · 3 citations · ⚡1
Preprint Oct 2026

Surrogate Uncertainty for Black-box Visual Grounding

Visual grounding is crucial for robots to understand language instructions with visual observations. This paper addresses the problem of visual grounding uncertainties, where a robot uses a black-box vision-language model (VLM) to ground free-form natural-language instructions to target objects. Prevailing methods deri...

Han-Bo Zhang, Tian-Run Hu, Cedar Site Bai et al. · 0 citations
Preprint Oct 2026

QVR3D: Query-Guided Visual Reasoning for Training-Free 3D Grounding

QVR3D is introduced, a query-guided visual reasoning framework that operates on RGB-D observations using off-the-shelf foundation models that integrates instance confidence, attribute consistency, and relationship consistency and supports both single- and multi-object grounding without task-specific adaptation.

Jung-Hao Kao, Jing Huang, Jia-Wei Liao et al. · 0 citations

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.