Preprint

BVS: Bayesian Visual Search with Multimodal Large Language Model for Fine-grained Perception

Geng Li Yuxin Peng

Jul 2026 · 0 citations · 35 references

Computer Science

TL;DR

BVS bridges prior guidance with posterior correction: it utilizes an early-stop attention rollout of MLLM to construct reasoning-aware priors, while employing a scale-aware non-stationary kernel and GP-UCB to dynamically rectify noise and recover missing information in the prior through iterative local observations.

Abstract

While Multimodal Large Language Models (MLLMs) demonstrate impressive general capabilities, they struggle with fine-grained perception in ultra-high-resolution (UHR) images, particularly for tiny objects in cluttered scenes. Existing methods face a dilemma: they either rely on inefficient prior-free scanning, or depend on static prior-driven heuristics that lack posterior correction to rectify initial model biases. To address this, we propose BVS (Bayesian Visual Search), a framework that formulates perception as a global optimization problem over a continuous spatial-scale manifold. Specifically, BVS bridges prior guidance with posterior correction: it utilizes an early-stop attention rollout of MLLM to construct reasoning-aware priors, while employing a scale-aware non-stationary kernel and GP-UCB to dynamically rectify noise and recover missing information in the prior through iterative local observations. We provide theoretical guarantees via sub-linear regret bounds, and extensive experiments demonstrate that BVS significantly outperforms state-of-the-art baselines with a superior trade-off between accuracy and efficiency.

arXiv Semantic Scholar

View source

Similar papers

Jul 2026

Seeing or Knowing? Visual Context Sensitivity in Multimodal Large Language Models

For the coarse attributes the authors study, MLLMs encode the visual evidence but cannot reliably control their reliance on it, indicating that for the coarse attributes they study, MLLMs cannot reliably control their reliance on it.

Jiaang Li, Chengzu Li, Zhaochong An et al. · arXiv.org · 0 citations

Preprint Aug 2026

Perception Before Supervision: Self-Contained Visual Distillation from Counterfactual Blind Spots

This work introduces CVPD (Contrastive Counterfactual Visual Process Distillation), which is the first fully self-contained framework for dense, on-policy, token-level visual self-distillation for MLLMs, and proposes a three-gate Counterfactual Criterion that identifies visual blind spots where zooming into a region ch...

Shravan Venkatraman, Omkar Thawakar, Ritesh Thawkar et al. · 2 citations

Jul 2026

Beyond Single Expert: Harmonizing Diverse Visual Priors in MLLMs for Spatial Understanding

ViPS introduces an Efficient Prior Proxy to generate multiple foundational priors with minimal inference overhead, and a Dynamic Prior Fusion mechanism to achieve harmonious and context-aware prior fusion and injection from the prior proxies.

Xiaofan Lin, Xiaohu Huang, Kai Han · arXiv.org · 0 citations

Preprint Aug 2026

Falcon Perception-HD: High Density Perception via Reinforcement Learning

This paper explores post-training reinforcement learning (RL), specifically GRPO, to directly align autoregressive perception models with their evaluation metrics, and designs an RL framework that addresses perception-specific challenges: reward design for set-structured outputs and multi-head sampling control.

Sofian Chaybouti, Yasser Dahou, N. Huynh et al. · 0 citations

Preprint Aug 2026

Think with Structured Grounding: Perceptual Reinforcement Learning for Chart and Visual-Tabular Understanding

Think with Structured Grounding (TwSG), a novel fine-grained image perception framework designed to internalize complex images's tool-use capabilities within the model, is proposed, endowing models with native fine-grained region description and flexible reasoning capabilities.

Chang-Jiang Jiang, Qian-Nian Zhao, Lei Xin et al. · 2 citations

Preprint Jul 2026

Vision Pretraining for Dense Spatial Perception

This work proposes masked boundary modeling, a self-supervised paradigm that dynamically learns sub-pixel boundary representations and subsequently leverages the discovered boundary-bearing tokens as masked targets to facilitate dense visual token learning.

Zelin Fu, Bin Tan, Chang Sun et al. · 3 citations · ⚡2