MDGS-Net: a multi-defect geometry-enhanced network for instance segmentation of subway tunnel defects

Abstract

Surface defect inspection in subway tunnels remains challenging due to multiple defect types, irregular geometric structures, blurred boundaries, and significant scale variations, which limit the accuracy of existing instance segmentation methods. This paper proposes MDGS-Net, a Multi-Defect Geometry-Enhanced Segmentation Network, based on the Ultralytics YOLO11 framework for accurate and efficient multi-defect instance segmentation in subway tunnels. The proposed network incorporates geometry enhancement, spatial prior guidance, multi-scale feature fusion, and boundary refinement strategies to improve defect representation and segmentation performance. Specifically, a Directional Geometry-Enhanced Convolution C3K2 Module and weighted convolution are introduced into the backbone network to strengthen the feature extraction capability of slender and irregular defects. Meanwhile, a Gaussian Prior Attention module is designed to exploit spatial prior information and enhance the localization of defect regions. In the neck network, a coordinate-guided feature fusion module is developed to improve multi-scale feature aggregation, while a Boundary Feature Refinement Segmentation Head is proposed to enhance boundary localization and preserve fine-grained defect details. Experimental results demonstrate that MDGS-Net achieves 95.4% mAP50 for detection and 94.3% mAP50 for segmentation, outperforming the baseline YOLO11n-seg by 3.4% and 3.7%, respectively. Furthermore, the proposed model maintains an efficient architecture with only 3.287 M parameters and 9.1 GFLOPs, achieving a favorable balance between accuracy and computational cost. The proposed method provides an effective and efficient solution for intelligent subway tunnel defect inspection and has potential for practical deployment in tunnel maintenance scenarios.