TIPSter: Zero-Shot Anomaly Detection via Language-Steered Anomaly-Informed Patches

Abstract

Industrial anomaly detection is essential for automated quality inspection, yet collecting representative defect samples for every product and failure mode remains costly and often impractical. In this article, we propose TIPSter, a Zero-Shot Anomaly Detection (ZSAD) framework that leverages language-steered anomaly-informed patches to identify defects without requiring task-specific training data. The proposed approach decomposes each image into local visual regions and aligns them with textual descriptions of normal and defective industrial patterns, enabling the precise detection of subtle surface irregularities, structural flaws, and texture deviations. By integrating patch-level visual evidence with the semantic generalization capabilities of Vision-Language Models (VLMs), our approach improves both image-level anomaly detection and spatial defect localization across multiple industrial datasets, yielding a 3.3-point improvement in average pixel-level F1 score over state-of-the-art methods. Furthermore, we thoroughly evaluate the model on the real-world production line of the Industrial Computer Engineering Laboratory (ICE Lab) at the University of Verona across two distinct product categories, thereby validating its practical zero-shot efficacy in manufacturing scenarios involving new products, unseen defects, and limited annotations. The code is available at https://github.com/intelligolabs/TIPSter.