Jan 2025· IEEE Transactions on Geoscience and Remote Sensing· Vol 64, pp. 5632017-5632017· 12 citations· ⚡ 1 influential· 81 references
Computer Science
TL;DR
DeepKANSeg, a novel network based on the Kolmogorov–Arnold network (KAN), achieves superior performance in terms of accuracy compared to state-of-the-art methods and improves interpretability, making it suitable for explainable learning in remote sensing.
Abstract
Semantic segmentation is vital for remote sensing applications, where accurate feature extraction and representation are essential. Existing encoder–decoder architectures often fail to fully utilize high-dimensional features and recover fine details during decoding. To address this problem, we propose DeepKANSeg, a novel network based on the Kolmogorov–Arnold network (KAN). KAN’s ability to decompose complex functions into univariate transformations enables flexible modeling of intricate data patterns. Our approach introduces two key innovations: a KAN-based deep feature refinement (DFR) module composed of DeepKAN to capture complex spatial and semantic relationships from high-dimensional features, and a global–local KAN (GLKAN) module replacing the traditional multilayer perceptron (MLP) layers with KAN-based linear layers to enhance fine-grained decoding. To evaluate the effectiveness of the proposed method, extensive experiments are conducted on two well-known fine-resolution remote sensing benchmark datasets, namely ISPRS Vaihingen and ISPRS Potsdam. The results demonstrate that the KAN-enhanced segmentation model achieves superior performance in terms of accuracy compared to state-of-the-art methods. Moreover, the univariate decomposition improves interpretability, making it suitable for explainable learning in remote sensing. The source code for this work will be accessible at https://github.com/sstary/SSRS
Vision foundation models (VFMs) pretrained on large-scale datasets have significantly improved performance in remote sensing semantic segmentation. However, existing methods typically rely on full fine-tuning, which requires updating all model parameters. Instead of updating the full parameter set, parameter-efficient fine-tuning (PEFT) achieves competitive performance by optimizing only a small subset of parameters. Despite its success, most existing PEFT methods are mainly designed for natural image tasks and fail to account for the unique multiscale characteristics of remote sensing images. To address these challenges, we propose multi-scale cognitive feature refinement (MsRE) tuning, a novel PEFT method tailored for remote sensing semantic segmentation. In particular, MsRE captures multiscale contextual information by applying cognitive operations with different cognitive fields to intermediate features of the backbone. It then introduces a set of learnable tokens to establish interactions with features at different scales, enabling precise feature refinement and progressive feature propagation across network layers. This mechanism enhances the model’s ability to understand complex remote sensing scenes and improves downstream segmentation performance. With significantly fewer trainable parameters, MsRE provides an efficient yet effective solution for adapting VFMs to remote sensing segmentation tasks. Extensive experiments demonstrate that MsRE achieves competitive segmentation performance with substantially fewer trainable backbone parameters, providing a favorable balance between accuracy and parameter efficiency. The project is available at http://woldier.top/MsRE
Bin Wang, Shun Lv, Zhi Li et al.· IEEE Transactions on Geoscie...· 0 citations
The semantic interpretation of remote sensing imagery through segmentation has become indispensable for a wide range of applications, including resource exploration, environmental assessment, and land-use analysis. Yet, accurate parsing of such images remains challenging because complex object boundaries and large scale differences often weaken the ability of conventional Convolutional Neural Network (CNN)-based methods to preserve local details. In response, this study constructs a segmentation framework that couples wavelet convolution with the Mamba architecture. To strengthen feature learning in the intermediate stages, an Auxiliary Segmentation Module (ASM) is employed to provide additional supervisory guidance, which supports optimization and encourages the representation of subtle semantic details. Wavelet-transform convolution is also introduced into the downsampling path, enabling spatial cues and frequency-related information to be exploited in a more coordinated manner for finer boundary and texture modeling. Experiments on public remote sensing datasets and mining area imagery further confirm the effectiveness of the method. Compared with several existing segmentation approaches, the proposed model delivers better overall performance in mIoU, F1-score, and recognition accuracy, particularly in scenes where multiple land-cover categories are heavily interlaced. Moreover, these gains are obtained with relatively low model complexity, suggesting good potential for practical deployment in land monitoring and ecological management.
Wenxi He, Zongmin Yin, Yulong Yang et al.· Remote Sensing· 0 citations
Semantic segmentation, assigning a class label to each pixel, has been revolutionized by deep neural networks. A significant milestone, the Fully Convolutional Network (FCN), demonstrated that a purely convolutional architecture could outperform previous approaches. Subsequent architectures largely adopted its encoder–decoder pattern. Models like U-Net and the DeepLab series advanced convolutional segmentation, and transformer-based designs such as SegFormer and EfficientViT achieved even higher accuracy. However, remote sensing imagery, often captured from an overhead view, differs significantly from the social media-like datasets on which these models are typically benchmarked. Remote sensing imagery also introduces the challenges of multi-scale objects, varying resolutions, complex scenes, extreme background/foreground imbalance, large intra-class variation, and small inter-class variation. Specialized architectures (FarSeg, AerialFormer) and benchmarks (iSAID) exist, but remote sensing remains challenging for segmentation. One approach for improving performance in remote sensing imagery is the use of the Differential Morphological Profile (DMP). Initially developed for satellite image segmentation, the DMP encodes explicit shape information for a given input image via the application of grayscale morphological operations. It has been incorporated into deep neural networks called DMPNets which take as input this shape information and produce features for tasks such as classification and object detection of overhead remote sensing imagery. This dissertation extends DMPNet to semantic segmentation of remote sensing imagery. A Direct-In integration strategy, in which the input image is converted to grayscale and combined with its morphological profile, consistently underperforms RGB-only baselines across six architectures and three benchmark datasets, with an ablation study confirming that the loss of color information is the primary cause. Applying the Hybrid DMP dual-stream design from prior DMPNet work, which processes RGB and DMP inputs through parallel encoders before feature fusion, substantially recovers this deficit and in many configurations surpasses the RGB baseline, with the largest gain of +3.19 mIoU observed for SegNeXt-S on the ISPRS Potsdam dataset. Several alternative fusion strategies beyond the original concatenation method are evaluated, including element-wise operations and squeeze-excitationbased attention, with squeeze-excitation methods emerging as the most consistently competitive across architectures and datasets. Per-class analyses reveal that object classes with strong geometric regularity, such as roundabouts, cars, and agricultural fields, benefit most from DMP integration, while classes whose primary discriminative cue is spectral appearance, such as swimming pools and barren land, are most harmed by the loss of color information under the Direct-In strategy. Experiments with remote sensing-specific architectures show that these specialized designs already capture much of the shape information the DMP provides, yielding more modest Hybrid DMP gains than for general-purpose architectures. Notably, general-purpose architectures enhanced with Hybrid DMP are competitive with, and in some cases exceed, the purpose-built remote sensing architectures. An extension to instance segmentation via Mask-RCNN reveals that the DMP provides complementary but not dominant signal in detect-then-segment pipelines, where spatial cropping attenuates the globally coherent morphological features. Together, these findings establish that DMP features offer useful complementary shape information for remote sensing segmentation, but that preserving RGB information through a dual-stream design is critical to realizing this benefit.
Semantic segmentation of remote sensing images (RSIs) often struggles with boundary blurring, structural discontinuity, and category confusion due to limitations in conventional interpolation and dynamic upsampling methods. This article proposes RSUS, a new upsampling layer designed to preserve semantic consistency and spatial structure. RSUS consists of three components: 1) global context vector aggregation (GCVA) for content-aware prediction kernels that introduce global priors into local feature reconstruction; 2) cross-scale anisotropic implicit positional encoding (CAIPE) for direction-sensitive spatial deformation and structural alignment; and 3) adaptive high-frequency structure gating (AHSG) to enhance boundary-related frequency responses. In addition, persistent homology (PH) is used as a topological analysis tool to assess connectivity and structure preservation beyond pixel-level metrics. Extensive experiments on five datasets (International Society for Photogrammetry and Remote Sensing (ISPRS) Vaihingen, ISPRS Potsdam, LoveDA, UAVid, and Xining) demonstrate that RSUS improves performance across mainstream segmentation networks, outperforming advanced upsampling methods. Analyses show RSUS alleviates structural misalignment, improves category consistency, and recovers fine-grained boundaries in remote sensing segmentation. The source code is available at: https://github.com/Ronin-711/RSUS
Yaning Liu, Ronghao Yang, Shaoda Li et al.· IEEE Transactions on Geoscie...· 0 citations
Semantic segmentation of high-resolution remote sensing images remains challenging due to complex spatial structures, multiscale object variations, fine-grained category differences, and high interclass similarities. Conventional segmentation methods usually rely on fixed convolutional heads or single feature representations, which makes it difficult to effectively model both intraclass appearance variations and interclass texture similarities, often leading to category confusion, missed objects, and incomplete segmentation in complex scenes. To address these challenges, we propose a state-aware prototype learning network, termed SAPLNet. Specifically, a cross-stage state refiner is introduced to progressively refine multilevel features by integrating the input features with the outputs of different stages through state-aware gated normalization. Then, a weighted feature pyramid decoder performs top-down fusion of the refined hierarchical features, combining high-level semantic information with low-level spatial details. Furthermore, a state-aware multiprototype classifier is designed to construct multiple semantic prototypes for each class via ground-truth-guided local class-center extraction and momentum-based prototype memory updating. A global state vector derived from the refined cross-stage features is used to adaptively modulate decoder features, improving the matching reliability between pixel features and class prototypes. In addition, prototype compactness loss, prototype diversity loss, and lightweight boundary loss are employed to enhance intraclass consistency, prototype discriminability, and boundary awareness. Experimental results demonstrate the effectiveness and superiority of SAPLNet.
Zeyu Zhao, Zhaolong Gao, Jun Feng· IEEE Journal of Selected Top...· 0 citations
Multimodal semantic segmentation of high-resolution remote sensing imagery is important for fine-grained land-cover interpretation. However, existing fusion methods still suffer from unstable shallow optical-DSM alignment and deep feature degradation caused by heterogeneous frequency noise, boundary-detail loss, and inconsistent spatial responses. To address the aforementioned challenges, this letter proposes a coarse-to-fine progressive fusion network (CFPFNet). Specifically, a visual state space model extracts a Mamba-derived global structural prior to guide the coarse-grained context enhancement (CGCE) module for preliminary cross-modal alignment. Then, the fine-grained adaptive frequency-spatial fusion (FGAF) module performs amplitude-phase collaboration and adaptive spatial cross-gating for multiscale semantic refinement. Experiments on the ISPRS Vaihingen and Potsdam datasets demonstrate the effectiveness of CFPFNet. On Vaihingen, CFPFNet improves mIoU and mF1 by 2.22% and 1.39% over the simple dual-stream baseline, respectively.
Di Zhang, Yuhang Yan, Q. Niu et al.· IEEE Geoscience and Remote S...· 2 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.
