SAM-STF: A Multibranch SAM-Guided Spatial–Temporal-Frequency Aware Framework for Semantic Change Detection

Abstract

Semantic change detection (SCD) aims to determine whether land-cover categories change and how they evolve across multitemporal remote sensing images. Despite recent advances, existing methods still face some challenges. First, the encoders of visual foundation models produce overly smoothed representations, weakening fine-grained boundary information that is essential for object-level changes. Second, common temporal modeling strategies, such as feature differencing or fusion, fail to capture bitemporal interactions and recognize subtle changes. Third, relying on a single backbone often fails to achieve a tradeoff between global semantic consistency and local spatial precision. As such, we propose a novel multibranch segment anything model (SAM)-guided spatial–temporal-frequency aware framework (SAM-STF) for SCD. It adopts a two-stream Siamese encoder, where the frozen SAM branch is introduced to capture global semantic context and boundary priors, while the lightweight ResNet34 branch delivers fine-grained local spatial details. In particular, we design a sample-specific low-rank modulation (S2LoRM) module to provide input-conditioned low-rank residual modulation for the SAM branch. To enhance the boundary clarity of the changed areas, we design an adaptive Gaussian-derivative projection (AGDP) module that refines shallow SAM features through multidirectional Gaussian derivative filtering. In addition, a bidirectional temporal-frequency awareness (BTFA) module is developed to symmetrically model bidirectional temporal interactions while integrating multiscale spatial context and frequency-domain information. Therefore, subtle and complex bitemporal changes can be located and detected, offering more precise boundary description and stronger robustness to complex spatial–temporal scenarios. Extensive experiments on three benchmark datasets demonstrate that SAM-STF achieves the best performance, with a separated kappa (SeK) of 65.92% on the Landsat-SCD dataset, 24.67% on the SECOND dataset, and 48.02% on the JL1H dataset, respectively. The code will be available at https://github.com/SUPERMAN123000/SAM-STF