VeriFuse assigns the VLM a clear and constrained role in cooperative perception: semantic reasoning resolves ambiguity among cross-agent hypotheses, while deterministic constraints determine the final 3D geometry.
Abstract
Vision-language models (VLMs) have demonstrated strong scene understanding and semantic judgment across diverse tasks, but their appropriate role in cooperative perception remains unclear. Directly asking a VLM to regress 3D detections is unreliable and computationally expensive, whereas using it to select the output of a single source discards useful information from other agents. We introduce VeriFuse, a bounded arbitration framework for vehicle-infrastructure cooperative 3D detection. Each agent first produces detections independently. Around each vehicle and roadside proposal, VeriFuse generates source-conditioned geometric candidates and combines the original detections, their perturbations, and cross-source hypotheses into a unified candidate pool. A frozen VLM then chooses among three admissible actions: SELECT an adequate candidate; REFINE an existing anchor when an object is supported but all candidates are geometrically inadequate; or REJECT an unsupported infrastructure-only proposal. Experiments on the DAIR-V2X dataset show that VeriFuse achieves 0.494/0.357 cooperative 3D AP50/AP70 and limits the relative vehicle-side BEV AP50 drop under a 300 ms delay to 1.7%. Overall, VeriFuse assigns the VLM a clear and constrained role in cooperative perception: semantic reasoning resolves ambiguity among cross-agent hypotheses, while deterministic constraints determine the final 3D geometry.
GPD (Geometry-Privileged Distillation) is proposed, which makes geometric evidence the privilege in on-policy self-distillation (OPSD), outperforming both GRPO and answer-privileged OPSD across spatial reasoning benchmarks.
Hong-Xing Li, Yi-Xin Li, Ding-Ming Li et al.· 0 citations
Vision-language models (VLMs) increasingly rely on point coordinates as a compact and executable interface for visual grounding in GUI interaction, robotic manipulation, and interactive visual systems. However, learning reliable pointing behavior remains difficult because the supervision space is inherently non-unique:...
Jing-Yang Su, Pu Cao, Xiu-Ze Jin et al.· 0 citations
A modular, predictor agnostic, tool-augmented framework that equips a small VLM with geometric tools: 3D object detection, metric depth estimation, and deterministic solvers for distance, size and bearing, matching a scripted pipeline on three of four tasks.
Autonomous aerial object retrieval requires reliable high-level decision-making particularly when perceptiondriven conditions govern task progression. Conventional Behavior Trees (BTs) are widely used to structure such missions, but their deterministic condition evaluations limit the use of complex semantic perception....
Mohammad Soltanshah, Abolfazl Eskandarpour, Kamal Gupta et al.· 2026 IEEE 22nd International...· 0 citations
GaussVLA is proposed, a Mamba-based VLA that incorporates two custom modules: Gaussian Spatial Tokenizer (GST) to lift frozen semantic and depth features into compact 3D Gaussian tokens, and Depth-Aware Chain-of-Thought (DA-CoT) that performs structured, non-autoregressive geometric reasoning under language and flow-ti...
MD SELIM SAROWAR, Md Tanvir Islam, Sungho Kim et al.· 1 citation
This work introduces FailBench, a benchmark for robot failure detection comprising 2,197 manipulation attempts across 14 public sources (12 real-world, 2 simulated), and shows that input-level intervention--spatially localizing and cropping outcome-relevant regions--improves the top detector by 2.4 percentage points wi...