Skip to content

Spectral Reversal: Counteracting Singular Value Bias for Graph Prompting

Sep 2026 · arXiv.org · Vol abs/2609.32143 · 0 citations · 48 references
Computer Science

TL;DR

Spectral Reverse Prompt (SRP), a prompting framework that rebalances the spectral contributions of frozen GNN encoders, achieves state-of-the-art performance with minimal additional parameters, highlighting that reweighting spectral components is a principled and effective strategy for parameter-efficient graph adaptation.

Abstract

Pre-training Graph Neural Networks (GNNs) via self-supervised learning has become a dominant paradigm, yet efficiently adapting frozen encoders remains a challenge. Graph prompting offers a parameter-efficient alternative to fine-tuning, but existing methods largely treat pre-trained models as opaque feature extractors, ignoring their internal spectral structure. In this work, we identify a systematic phenomenon in pre-trained GNNs, which we term spectral bias: optimization during pre-training disproportionately aligns representations with directions associated with large singular values, leaving low-energy directions under-explored. We show that these underutilized directions can encode complementary information that is beneficial for downstream adaptation, especially under distribution shift. To leverage this insight, we propose Spectral Reverse Prompt (SRP), a prompting framework that rebalances the spectral contributions of frozen GNN encoders. SRP applies a learnable soft-thresholding mask in the spectral domain to down-weight dominant directions while amplifying weaker ones. In addition, SRP incorporates a null-space augmentation module that captures variation in directions with minimal activation under the frozen encoder. Extensive experiments across multiple benchmarks demonstrate that SRP achieves state-of-the-art performance with minimal additional parameters, highlighting that reweighting spectral components is a principled and effective strategy for parameter-efficient graph adaptation.

View source

Similar papers

Sep 2026

Beyond Homophily: Spectrum-Based Graph Pre-Training and Cluster-Augmented Prompt Tuning

This paper presents a systematic spectral analysis of graph prompting, revealing the inherent limitations of spatial-domain prompts in capturing attenuated high-frequency signals and proposes SCA-GPPT, a spectrum-aware framework that unifies Spectrum-based Graph Pre-training and Cluster-Augmented Prompt Tuning.

Ting-Ting Li, Yong-Hao Li, Xiang-Kun Wang et al. · 0 citations
#artificial intelligence Preprint Sep 2026

Neuralyzing the Trace: Selective Representation-Level Unlearning with Contrastive Sparse Autoencoders

This work introduces SCALPEL, a contrastive sparse autoencoder designed to learn more selective forget features and shows theoretically that contrastive training promotes target-selective features and that the selection score controls expected background knowledge perturbation.

Itai Zehavi, Fanny Jourdan, Ulrich Aivodji · 0 citations
Sep 2026

Masked Swingers: Harnessing Data Augmentation to Advance Autoencoders for Self-Supervised Learning

Self-supervised learning (SSL) removes the need for annotations and makes models that are capable across more domains than supervised learning. The autoencoder SSL framework learns by reconstructing its own input after information loss through a bottleneck or noise injection. Masked autoencoders (MAE) are the most succ...

A. Fuller, Scott C. Lowe, Daniel G. Kyrollos et al. · 0 citations
Conference Open access Sep 2026

Learnable Data Augmentation and Contrastive Pre-training for Temporal Link Prediction

This paper proposes a novel model, ContraTGT, which employs a dual-view graph transformer, and puts forth an innovative, learnable data augmentation method, which involves selective masking of elements within dual-view sequences, thereby amplifying the potency of the contrastive learning approach.

Cang-Hong Jin, Jia-Feng Zhao, Feng Xu et al. · 0 citations
#machine learning Preprint Sep 2026

LILA: Calibration-Free Structured Pruning of Large Language Models via Latent Spectral Geometry

Extending LILA to dynamically allocate sparsity budgets via KS-scores yields state-of-the-art generative preservation at moderate compression, while uncovering fundamental single-layer architectural bottlenecks at higher compression regimes.

Sankar Behera, Dhruv Singh, Anshika Agnihotri et al. · 1 citation

Related blog posts

MIT News · Artificial Intelligence Sep 29, 2026

Who we become when we talk to machines

Professor Sherry Turkle’s new book, “Artificial Intimacy,” offers a withering critique of chatbots and the antisocial dynamics she believes they encourage.

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.