High-precision image segmentation based on a hybrid CNN–Transformer architecture

Abstract

To address the challenge of balancing long-range dependency modeling and detail fidelity in urban scenes, medical slices, and high-resolution remote sensing imagery, this study proposes a lightweight hybrid architecture that integrates a lightweight CNN with a window-based Transformer. In the front-end, depth wise separable convolutions and residual connections are employed to extract edge-sensitive features, complemented by an edge-guided branch and channel recalibration to enhance thin structures and sharp boundaries. In the intermediate stage, multi-scale local window self-attention is utilized to capture long-range dependencies, while learnable windows and sparse global proxies are introduced to reinject global semantic information. In the bridging stage, deformable alignment and gated residual connections are adopted for cross-scale feature fusion, with weights jointly modulated by edge density and global proxies. Experiments are conducted on the Cityscapes validation set under a single-scale inference setting with a resolution of 2048×1024, using an NVIDIA A100 80 GB GPU, batch size of 1, and mixed precision enabled. The proposed method achieves an IoU of 92.3, an F1 score of 93.1, and a speed of 25 FPS, only 12.8 M parameters, 31.6 GFLOPs, and peak GPU memory usage of 2.8 GB. Compared with U-Net (IoU 88.5, F1 89.2, 20 FPS) and SegFormer (IoU 90.1, F1 91.0, 18 FPS), the proposed approach demonstrates clear advantages in accuracy, real-time performance, and intrinsic structural efficiency. The results indicate that the collaboration between global semantics and local details within a unified weighting domain can effectively improve the separability and deploy ability of high-resolution segmentation.