Explainable Video Anomaly Detection Using Video Masked Autoencoders and Convolutional LSTM

Abstract

This paper presents a unified and interpretable framework for video anomaly detection that integrates a pretrained Video Masked Autoencoder (VideoMAE), a ConvLSTM-based temporal modeling module, and a 3D CNN decoder for spatiotemporal reconstruction. The model is trained exclusively on normal video sequences and identifies anomalies through reconstruction discrepancies.To provide comprehensive interpretability, the framework combines Grad-CAM for spatiotemporal localization, SSIM-based heatmaps for structural deviation analysis, and CLIP-based textual descriptions for semantic explanation of anomalous events. Extensive experiments on benchmark datasets, including UCSD Ped1, UCSD Ped2, and Avenue, demonstrate the effectiveness of the proposed approach, achieving frame-level AUC scores of 99.03%, 99.20%, and 90.40%, respectively. The results highlight the capability of the framework to deliver both accurate detection and multi-level explainability, supporting its applicability in practical surveillance systems.