Object Surface Defect Segmentation using Vision Transformers and Hybrid Models

Abstract

Detection of surface defects in the manufacturing industry is fundamental to maintain the quality of products and reduce waste production. Inspection processes are always affected by human error due to the nature of manual operations in manufacturing environments, particularly in cases of high production rates, while traditional machine vision systems cannot generalize over the complex structure of surface materials. In order to achieve the goal of automated surface defect detection, convolutional neural networks have been proposed. However, the local feature extraction abilities restrict these network from detecting long-distance dependencies in surface material structures and hence limits its ability to detect defect boundaries accurately. This research paper offers an empirical comparison between three deep learning frameworks for pixel-based surface defect detection: a baseline CNN architecture known as U-Net, Vision Transformer (ViT) based on patch-wise attention mechanism, and TransUNet that combines CNN and transformer architecture. The three deep learning models were compared following a unified approach of training and evaluation over three benchmark industrial surface defect segmentation datasets, i.e., MVTec AD, NEU Surface Defect Database, and DAGM 2007, where the Dice Coefficient and mean Intersection over Union are used as evaluation criteria. It is observed that U-Net performs best in terms of maximum mIoU of 0.75 at Epoch 26, ViT attains 0.72 with better generalization behavior at Epoch 41, and TransUNet achieves 0.67 at epoch 19.