Skip to content
Conference

On the Instability of Saliency Maps under Sparse Perturbations

Aug 2026 · International Conference on Multimedia Analysis and Pattern Recognition · pp. 91-96 · 0 citations · 24 references

Abstract

Understanding the reliability of model explanations remains a critical challenge in deep learning. Prior work has shown that saliency maps can be manipulated by optimizing the input using gradient-based methods, where gradients of the loss with respect to the input are computed to generate dense perturbations that alter the model’s explanations. However, the impact of sparse perturbation constraints remains underexplored, largely due to the difficulty of optimization arising from their discrete and non-differentiable nature. In this work, we pioneer the investigation of this problem by employing a gradient-free sparse adversarial attack. Our goal is to modify only a small subset of pixels in an input image to successfully attack the model and induce distortions in the saliency maps, such that the highlighted important regions are displaced. Experimental results on both general-domain and medical vision models show that altering only a tiny fraction of pixels is sufficient to drastically change the model’s predictions and the spatial distribution of attention in the saliency maps. Our study provides new insights into the interplay between robustness and explainability against sparse perturbation, and highlights risks in relying on saliency-based explanation, particularly in safety-critical applications. Source code is available at https://github.com/ELO-Lab/SparseAttack-To-Saliency-Map.

View source

Similar papers

Scalable Perturbation-Based Explanations via Tradeoff-Conditioned Smooth Masking

This work introduces a sampling-free, perturbation-based training framework based on continuous and differentiable masking that achieves competitive or superior attribution faithfulness compared to strong sampling-based baselines, while dramatically reducing computational cost and enabling substantially improved scalab...

Mehdi Naouar, Jens Rahnfeld, Yannick Vogt et al. · 0 citations
Conference Sep 2026

Boosting adversarial attacks with attention-guided luminance perturbation

Deep Neural Networks (DNNs) are vulnerable to adversarial examples generated by adding human-imperceptible perturbations to benign inputs. Moreover, the transferability of adversarial examples enables effective black-box attacks across different models. However, existing attack methods typically generate spatially dens...

Zi-Han Huang · 0 citations
Sep 2026

Adversarial Training for Pixel Diffusion

This work identifies direct output access to the image statistics being corrected as a key factor governing when adversarial post-training succeeds, the first systematic study of adversarial post-training for pixel diffusion.

Xin Lin, Zhi-Fei Zhang, Yu-Qian Zhou et al. · 0 citations
Open access Sep 2026

DWAT: Density-Weighted Adversarial Training for Robustness Beyond the Training Perturbation Budget

Density-Weighted Adversarial Training is presented, a plug-in framework that spreads training over a set of sampled ϵ-balls and reweights each candidate adversarial example by a Gaussian density of its distance from the benign sample, and it is proved that the population objective upper-bounds the worst-case adversaria...

Jie-Ying Huang, Rui-Ming Zhu, Jia Xu et al. · 0 citations
#artificial intelligence Preprint Sep 2026

Do Influence-Derived Data Perturbations Enable Machine Unlearning? A Controlled Study of Three Plausible Roles

Deep Perturbation Learning is evaluated in three roles in which prior work has positioned it for machine unlearning: a direct deletion signal (the strongest claim), a utility-preserving regularizer, and a warm start for adversarial unlearning.

Chen Wu, Chrispine Kambimbi, Qi Zeng et al. · 0 citations
#machine learning Preprint Oct 2026

Loss-Invariant Projections as Passive Probes of Learned Representations

Learned feature representations in neural networks often contain structure beyond that directly used by the final task output. We study this structure using $\textit{passive probes}$ that apply fixed, untrained, property-independent projections to representations as they evolve during training. We motivate this approac...

Akshay Chandrasekhar, P. Melnyk · 0 citations

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.