Skip to content

VOR-Bench: A Human Perception-Driven Benchmark for Video Object Removal

Sep 2026 · 0 citations · 66 references
Computer Science

TL;DR

This work presents the VOR Dataset (VORD), the first benchmark dataset providing both paired edited videos and graffiti masks, and introduces VOR-MDSM, the first perception-driven VLM-based scoring model specifically designed for mask-guided VOR.

Abstract

Despite its crucial role in video object removal (VOR), existing evaluation paradigms face two critical limitations: questionable references and a misalignment between tradi- tional metrics and human preference. To address these challenges, we introduce VOR- Bench, which advances VOR evaluation through three integrated components. First, we present the VOR Dataset (VORD), the first benchmark dataset providing both paired edited videos and graffiti masks. Its unique strength lies in a diverse data spectrum, which encompasses model-generated, tool-rendered, and camera-captured data, ensuring robust assessment across real-world scenarios. Second, we develop rMPAF, a realistic Motion- capable Paired-video Acquisition Framework. By combining the strengths of image- based object removal and fine-tuned video generation models, rMPAF automatically generates realistic, motion-coherent paired videos. Finally, we propose three evaluation dimensions and introduce VOR-MDSM, the first perception-driven VLM-based scoring model specifically designed for mask-guided VOR. It bridges the gap between arithmetic metrics and human perception by covering the essential visual attributes and matching nuanced human judgment. Extensive experiments demonstrate that VOR-Bench yields evaluation results that align closely with human perception, achieving a remarkable cor- relation (\r{ho}>0.9) with subjective assessments. We will release VOR-Bench along with its documentation to ensure full reproducibility.

View source

Similar papers

Oct 2026

Learning Video Exposure Correction: Large-Scale Real-World Dataset and Method.

Exposure correction aims to enhance visual data degraded by improper exposures, thereby improving perceptual quality. While substantial progress has been achieved in single-image exposure correction, its extension to videos remains largely underexplored. Applying frame- wise correction methods to videos often results i...

Si-He Zhang, Hui-Yuan Fu, Jin Liu et al. · 0 citations

Human Visual Attention Modeling and Content Understanding for 360◦ Environment

This thesis introduces the first dataset dedicated to individualized scanpath prediction in 360° imagery, featuring explicit mappings between user identities and scanpaths and reveals the spatiotemporal regularities of human viewing patterns in panoramic scenes.

Yujia Wang · 0 citations

VIF-Bench: Evaluating Visual Instruction Following in Multi-Reference Image Generation

VIF-Bench is introduced, a benchmark of 1,241 tasks designed to assess the edge of model capabilities in this joint setting in which multiple references must be composed under multiple and heterogeneous visual-instruction images, and uncover three findings.

Yuta Oshima, Masakazu Yoshimura, Masahiro Suzuki et al. · 0 citations
#artificial intelligence Preprint Oct 2026

SAVU-BENCH: A Real-World Benchmark for Spatial Audio-Visual Understanding

Spatial audio-visual understanding requires models to recognize not only what is present, but also where events occur and how they relate across modalities. Existing benchmarks often rely on simulated scenes, evaluate isolated spatial skills, and provide limited diagnostic insight into failure modes. We introduce SAVU-...

Yu Chen, Rui-Hang Liu, Yang-Guang Xu et al. · 0 citations
#artificial intelligence Preprint Oct 2026

AcrossVAM2.0: Conflict-Aware Point Supervision for Parameter-Efficient Video Diffusion Adaptation

Explicit point correspondence is attractive for adapting video generators: it describes how visible surfaces move, but can overwrite the appearance prior learned by a large pretrained model. We study this tension in a 5.01B-parameter text-to-video diffusion transformer. ACROSSVAM2.0 combines a confidence-weighted track...

Nan Wu, Yee-Hung Hong · 0 citations
Preprint Oct 2026

HuC-VideoMAE: Human-Centric Video Masked Autoencoding from synthetic data

Modern action recognition models rely on video transformers pretrained on massive collections of web-crawled videos, such as Kinetics-700. However, the use of such data raises ethical concerns, as subjects'consent is typically not obtained. Recent high-quality synthetic video datasets generated from motion-capture data...

Ricardo Pizarro, Roberto Valle, J. M. Buenaposada et al. · 0 citations

Related blog posts

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.