Weighted multi-scale and wavelet-enhanced Segment Anything Model for salient object detection

Abstract

Salient Object Detection (SOD) is concerned with isolating the visually most noticeable objects in an image via precise segmentation. Previous approaches, especially those that adapt the Segment Anything Model (SAM), often yield saliency maps that contain incomplete object masks, blurred boundaries, and a lack of fine-grained details. This issue is especially severe for scenes with objects of different scales or complex textures. We argue that these issues stem from three inherent limitations of existing adaptation strategies: (1) existing adapters rely on rigid multi-scale fusion strategies, lacking learnable cross-scale calibration to handle objects of diverse sizes, (2) simple feature concatenation ignores cross-level semantic correlations, (3) spatial-domain operations inevitably discard high-frequency details. The proposed WMW-SAM, a Weighted Multi-Scale and Wavelet-Enhanced SAM, is designed to handle these limitations for SOD. Specifically, we develop the Weighted Multi-Scale Adapter (WMSA), which utilizes learnable weighting across multiple receptive fields to calibrate features of different scales. Then, our Multi-level Feature Cross-fusion Module (MFCM) employs cascaded top-down cross-attention to facilitate deep interaction that connects high-level semantics with low-level details. Finally, we develop a Detail Enhancement Module (DEM) that leverages the Discrete Wavelet Transform (DWT) to explicitly extract and enhance high-frequency sub-bands. This operation effectively recovers sharp boundaries and intricate textures, which are often overlooked by spatial-domain operations. Extensive experiments on multiple benchmark datasets demonstrate the superior performance of our proposed WMW-SAM, which achieves accurate and detailed saliency predictions.