Skip to content
Open access

Learn the interactions: Weakly supervised video anomaly detection with human-object interactions

Sep 2026 · PLoS ONE · Vol 21 · 0 citations · 53 references
Medicine

TL;DR

A novel weakly supervised cross-domain framework that enriches feature representations by explicitly integrating semantic cues from Human-Object Interactions (HOI) and action dynamics and achieves competitive performance against state-of-the-art methods under weak supervision in terms of AUC.

Abstract

Weakly supervised video anomaly detection remains a challenging problem, primarily due to the scarcity of abnormal training samples and the lack of diverse feature representations, which hamper the learning of discriminative models. To address these issues, we introduce a novel weakly supervised cross-domain framework that enriches feature representations by explicitly integrating semantic cues from Human-Object Interactions (HOI) and action dynamics. Specifically, we model domain-specific features using two independent encoders and fuse them via a bi-directional cross-attention mechanism to obtain a unified embedding. To further alleviate class imbalance and enhance feature discrimination, our network is jointly optimized with classification loss, a cross-domain contrastive loss, and a clip-level focal loss. The proposed method is evaluated with various visual backbones, demonstrating its adaptability and robustness. Extensive experiments show that our approach achieves competitive performance against state-of-the-art methods under weak supervision in terms of AUC. Specifically, our proposed method achieves 98.33% AUC and 87.96% AUC on two benchmark datasets, i.e., ShanghaiTech and UCF-Crime, respectively.

Read PDF

Similar papers

2026

Reconstructive Visual Tuning for Weakly Supervised Video Anomaly Detection

Weakly supervised video anomaly detection (WS-VAD) presents a significant challenge in security video surveillance, as it aims to accurately identify anomaly frames in untrimmed videos with only video-level supervision. Several recent studies exploit vision-language pre-training models, e.g., CLIP, to take advantage of...

Shuang-Qing Zhang, Wei Xu, Yu-Qi Fang et al. · 2 citations
Review Open access 2026

Language Models in Video Anomaly Detection: A Dynamic Survey

Video Anomaly Detection (VAD) plays a critical role in intelligent surveillance and public safety by identifying rare, unexpected, or contextually abnormal events in complex video environments. Conventional deep learning approaches have substantially advanced VAD, but they remain largely driven by visual representation...

Furkan Mumcu, Lokman Bekit, Michael J. Jones et al. · 0 citations
Preprint Oct 2026

MS2CAST: A Video Anomaly Detection Framework via Multi-Scale Spatial Cross-Attention and Efficient Spatiotemporal Networks

Video anomaly detection (VAD) is a challenging task for surveillance systems due to the scarcity and diversity of anomalous events. While semi-supervised VAD approaches have advanced the field, many rely on computationally intensive optical flow networks to capture motion dynamics. Convolutional neural networks (CNNs)...

Ali Safaei, Mohammad Kazemi · 0 citations
Open access 2026

Multimodal DynaST: A Multimodal Framework With Dynamic Sample Selection and Temporal Context Aggregation for Weakly Supervised Video Anomaly Detection

Experimental results on UCF-Crime, ShanghaiTech, and XD-Violence demonstrate the effectiveness of multimodal DynaST, a multimodal framework for weakly supervised video anomaly detection, and the benefit of multimodal learning for robust anomaly detection.

Wen-Jian Zhang, Xin-Xin Fan, Qi-Wei Pan et al. · 0 citations
Conference Sep 2026

Multimodal deception detection via feature reconstruction and spatio-temporal consistency modeling

This paper introduces a novel multimodal deception detection framework centered on feature reconstruction and temporal consistency modeling that outperforms baseline models across multiple metrics, including Accuracy, F1-score, and AUC.

Yao-Dong Zhou, Zong-Yu Zhang, Chun-Hua Ren · 0 citations
Open access Sep 2026

ActNet: focus-aware multi-scale CNN for human activity recognition from images

This article proposes ActNet, a novel deep convolutional neural network architecture that combines multi-scale feature learning with a focus-aware attention mechanism to address the problem of recognizing human actions from still images.

Şafak Kılıç · 0 citations

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.