TL;DR
A novel large multimodal model applying vision experts for industrial anomaly detection (abbreviated as Myriad), which treats conventional IAD models as VEs and converts their anomaly maps into lightweight prompts that steer a frozen Q-Former toward suspicious regions, while a compact low-rank adapter shapes features for IAD.
Zero-shot industrial anomaly detection (ZIAD) aims to develop a unified model capable of directly identifying unseen anomaly categories in images without requiring reference samples. Recently, large-scale Vision-Language Models (VLMs) such as CLIP have shown great potential for solving this task. However, existing methods typically rely on manual text prompts to guide VLMs in anomaly detection, which often fail to capture fine-grained semantic cues, leading to limited accuracy. To address the challenge, this paper proposes a novel Cross-Modal Guidance Learning (CMGL) framework for ZIAD. Instead of handcrafted textual prompts, CMGL introduces learnable prompting mechanism to fully exploit the collaborative guidance between visual and textual modalities for efficient unseen anomaly detection. Leveraging the frozen image encoder of pre-trained CLIP, the CMGL extracts multi-scale patch tokens and global tokens of the input image as visual representations. Then, informed by the cross-modal information, adaptive prompt vectors are constructed to obtain textual representations. In the process, a Learnable Context Block (LCBlock) and a Multi-Layer Perceptron (MLP) are introduced to extract holistic semantics and fine-grained details, and an Adaptive State Vector Module (ASVM) is designed to learn generalized normal and abnormal state vectors from extensive text descriptions. By aggregating the outputs of these components, textual representations of the image are acquired through the frozen text encoder. Finally, a Local-Global Token Integrator (LGTI) and an Uncertainty-Aware Anomaly Fusion Module (UAFM) are proposed to achieve anomaly recognition and localization through visual–textual alignment. Extensive experiments on multiple industrial datasets demonstrate the superiority of our method. Note to Practitioners—This paper presents a Cross-Modal Guidance Learning (CMGL) framework to address anomaly detection under the zero-shot setting. Unlike previous approaches that rely on manually crafted text prompts, the proposed CMGL derives task-relevant prompt cues from cross-modal data by the designed learnable prompting mechanism, guiding the model to automatically recognize and localize unseen anomaly categories without requiring any reference samples. Extensive experiments demonstrate the effectiveness and strong generalization capability of the proposed approach. Benefiting from these properties, our method provides a novel and effective ZIAD solution for identifying potential anomalies in real-world industrial scenarios where data distributions are uncertain or anomaly-related information cannot be clearly specified. Our project page is publicly available at https://aicoder12.github.io/CMGL/
Anomaly detection is a critical task with broad applications in industrial and medical domains. In this paper, we propose LCPCLIP, a novel model designed to address zero-shot anomaly detection by leveraging the power of learnable complementary anomaly prompts. Unlike traditional approaches that rely on pre-defined category labels or extensive manual prompt, LCPCLIP introduces multiple learnable anomaly prompts optimized through complementary anomaly prompt loss, which can ensure orthogonality and diversity in learned features. Meanwhile, it incorporates a Dynamic Category Feature Embedding (DCFE) module, which fuses category and global feature information into text prompts without requiring explicit category labels. Moreover, a Vision-Text Cross-Attention (VTCA) module is proposed to enhances the interaction between text prompts and fine-grained local features. Extensive experiments conducted on ten real-world industrial anomaly segmentation datasets demonstrate the superior performance of LCPCLIP in detecting anomalies without any prior knowledge of the dataset categories. It sets a new benchmark for adaptability and generalization in industrial applications. Our code and pre-trained models are available at https://github.com/rougasuki/LCPCLIP .
Industrial anomaly detection is a critical component of modern manufacturing. Most traditional unsupervised methods rely on modelling normal feature distributions, inherently limiting generalization to unknown categories. To improve generalizability, some recent methods incorporate vision-language models (VLMs) for zero-shot detection via text prompts. However, we observe that reasoning-oriented post-training can cause anomaly discrimination to collapse, with some fine-tuned models performing worse than their base VLMs. Existing methods also provide only textual decisions or coarse boxes, without pixel-level segmentation. A more explicit detection principle comes from human inspection: anomalies are identified by comparing a query image with a defect-free reference. Inspired by this, we propose InspectorGPT, a VLM framework centered on comparative reasoning. Given a normal reference and a query image, InspectorGPT compares them to identify discrepancies and perform multiple inspection tasks with detailed reasoning. We internalize this capability through Chain-of-Thought (CoT) fine-tuning and Group Relative Policy Optimization (GRPO) with tailored, verifiable rewards. We further introduce InspectorGPT-Seg for pixel-level anomaly masks. Segmentation supervision improves anomaly discrimination but weakens semantic reasoning, while joint training fails to balance them. We therefore train the two branches separately and combine them through task-vector fusion. Extensive experiments demonstrate superior multi-dimensional performance and generalization to unseen benchmarks, validating comparative reasoning for comprehensive industrial inspection.
The problem of industrial surface anomaly detection is still a disheartening one because there are light defect data, computationally intensive, and interpretable only to a limited extent when applied in real-world systems. In this paper, we present the HCG-MLLMAD - Hierarchical Cross-Modal Grounding framework of Multimodal Large Language Model-based Anomaly Detection - a variant of the original Light-MLLMAD architecture with multi-scale visual feature pyramids, prompt routing, and a hierarchical cross-attention fusion mechanism. The suggested architecture uses a two-branch visual encoder with ViT-Tiny (Vision Transformer-Tiny) and MobileNet-V4 representations, which allows extracting global structural semantics and fine-grained local texture features complementary to each other. A new Adaptive Contrastive Prototype Memory (ACPM) module updates normal-class prototypes dynamically at inference time to maintain high discrimination accuracy in the face of distributional shifts typical of changing manufacturing lines. A Hierarchical Prompt Router (HPR) module with three semantic granularities coarse, moderate and fine is chosen with the help of a lightweight Hierarchical Prompt Router (HPR) to create finer signals of cross-modal alignment. Extensive results on MVTec-AD, VisA, BTAD, and PCB-Defect benchmarks indicate that HCG-MLLMAD is more accurate and high-performing with 99.1% and 0.993 AUC respectively with one-shot supervision, compared to Light-MLLMAD [1] in terms of accuracy and F1-score. This model has a small 18.3 M parameter footprint and 78 ms inference latency on Jetson Orin Nano. Semantically grounded anomaly localization is verified by SHapley Additive exPlanations (SHAP)-based interpretability analysis and Gradient-weighted Class Activation Mapping (Grad-CAM++] visualizations, and is consistent with operator-defined defect vocabularies.
Vision-based industrial anomaly detectors are calibrated on one distribution but may be deployed on another that differs in illumination, fixture placement, or sensor characteristics, sharply degrading an otherwise accurate detector. Adapting to the incoming lot is a natural response, but labeled anomalies are scarce. We therefore consider calibration using only a handful of verified-normal images available before scoring the rest of the lot. Existing fixes require backpropagation, detector-specific tuning, or choices about feature directions that few calibration samples cannot justify. We present SPARC, a few-shot calibration method that intercepts patch features between encoder and detector and removes a closed-form, spatially indexed estimate of deployment-time nuisance through per-cell subspace projection. It needs only $k \le 8$ verified-normal images and uses the algebraic saturation rank $r{=}k{-}1$ on the encoder's native patch grid. The correction requires no gradient or weight updates and works with memory-bank, density, prototype, and mutual detectors. On the shift-prone benchmarks, SPARC improves pooled Image AUROC and AU-PRO$_{0.3}$ for all seven detectors whose image scores depend on corrected patch features by $+13.8$ and $+3.5$ percentage points (pp), respectively; on benchmarks without engineered shift, the changes are small and mixed. Controls that give competing corrections the same calibration images attribute these gains to the per-cell subspace structure rather than the images alone. Further ablations support the saturation-rank choice and characterize sensitivity to backbone and calibration conditions.
We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.
