Industrial anomaly detection via RGB-event fusion with multimodal large language models

Abstract

In complex industrial scenarios, traditional Industrial Anomaly Detection methods often perform poorly under extreme conditions such as low light, high-speed motion, and strong light interference. To address this, we propose RGB-Event Industrial Anomaly Detection via Vision-Language Model (ReIAD-VL), integrating RGB images and event streams with a multimodal large language model. RGB images provide appearance, texture, and structural information, while event streams capture asynchronous brightness changes and high-temporal-resolution motion cues. To reduce temporal and representational discrepancies between RGB and Event data, we propose a spatio-temporal alignment and fusion module that performs deep alignment and fusion of RGB-Event features, improving robustness and semantic consistency. To further tackle misaligned perception and diagnostic hallucinations of tiny defects in industrial scenarios by general visual-language models, we propose a fused anomaly information optimizer to enhance anomaly localization and discrimination. We also construct an RGB-Event IAD dataset as a benchmark with diverse scenarios. System performance from representation to reasoning improves through a three-stage training pipeline for cross-modal feature alignment and language generation. Experiments show ReIAD-VL consistently outperforms mainstream multimodal models on anomaly detection and language generation metrics, demonstrating strong generalization, interpretability, and industrial adaptability. This framework provides an effective solution for integrated anomaly perception, understanding, and reasoning in industrial scenarios.