Skip to content
Open access

Hybrid SNN-Transformer Network With Membrane-Potential-Aware Attention for Hyperspectral Image Classification

2026 · IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing · Vol 19, pp. 30777-30794 · 0 citations · 40 references

TL;DR

A spiking grouped separable convolutional vision transformer (Spiking-GSCViT) featuring a novel membrane-potential-aware grouped self-attention (MPA-GSA) module that mitigates the data-hungry nature of both ViTs and SNNs, enabling accurate and energy-efficient HSIC under limited supervision.

Abstract

Vision Transformers (ViTs) achieve high accuracy in hyperspectral image classification (HSIC) yet incur prohibitive computational cost, hindering edge deployment. Spiking neural networks (SNNs) promise energy-efficient event-driven computation, but directly transferring ViTs to SNNs leads to severe information loss from binary spikes, which is further amplified under the extremely limited labeled samples typical of HSIC. To address these intertwined challenges, we propose a spiking grouped separable convolutional vision transformer (Spiking-GSCViT) featuring a novel membrane-potential-aware grouped self-attention (MPA-GSA) module. MPA-GSA calculates attention weights from continuous, differentiable membrane potentials, rather than from lossy binary spikes. Moreover, leveraging the time-varying nature of membrane potentials, this novel attention mechanism is able to attend to different spectral–spatial regions across multiple time steps, thereby preserving rich discriminative features. Extensive experiments on four benchmark datasets with only 1% to 5% of the labeled samples per class show that Spiking-GSCViT matches or surpasses its ANN counterpart (94.16% on Indian pines and 97.71% on Salinas), with only 0.21 M parameters. The consecutive MAC operations are reduced by 81%, and the spike-driven synaptic operations (SOPs) are decreased by 39.2%. As a lightweight SNN-ViT hybrid architecture, Spiking-GSCViT simultaneously mitigates the data-hungry nature of both ViTs and SNNs, enabling accurate and energy-efficient HSIC under limited supervision.

Read PDF

Similar papers

2026

Label-Efficient Hyperspectral Image Change Detection via Multiscale Self-Distillation

Methods based on deep learning have achieved remarkable success in hyperspectral image change detection (HSI-CD). However, the high dependence on massive labeled data and the computational complexity of state-of-the-art models pose significant challenges, particularly in scenarios with extreme label scarcity. To addres...

Zhi-Yuan Zhang, Zhi-Jing Ye, Cheng-Yong Zheng et al. · 0 citations
Open access Sep 2026

SpikeYOLO-C2TCA: An Improved Spiking Convolutional Neural Network for Remote Sensing Image Object Detection

SpikeYOLO-C2TCA, an improved SNN detector built upon the SpikeYOLO backbone, delivers better detection accuracy than the baseline while consuming less energy, offering a practical lightweight solution for edge platforms such as on-orbit satellites and long-endurance unmanned aerial vehicles.

Unknown authors · 0 citations
Open access Oct 2026

Hybrid transformer and CNN for hyperspectral image classification with spectral band selection

A new method for HSI classification by integrating spectral band selection with advanced deep learning techniques is presented, and an optimized band selection method alongside data augmentation using crop-and-resize transformations to enhance feature diversity is proposed.

Eman N. Abdelhafez, Ahmed Hagag, Tamer A. Abassy · 0 citations
Review Open access Sep 2026

Systematic Review on Artificial Intelligence‐Enhanced Hyperspectral Imaging for Food Processing: Applications, Technical Bottlenecks, and Future Perspectives

Hyperspectral imaging (HSI) integrates spatial and spectral analysis for nondestructive food quality monitoring. While the integration of artificial intelligence (AI) has significantly advanced HSI analytics, prior reviews have predominantly focused on static, postharvest quality grading. This leaves a critical gap r...

Xiao-Wei Huang, Bao-Ze Liu, Xiao-Dong Zhai et al. · 0 citations
Open access 2026

FlashViT: A Memory-Efficient and Explainable Vision Transformer for Multi-Class Sweet Orange Disease Classification

Accurate and computationally efficient plant disease detection is essential for scalable precision agriculture systems. While convolutional neural networks (CNNs) have achieved strong performance in crop disease classification, their localized receptive fields may limit modeling of long-range spatial dependencies in co...

Shahrin Khan, Sadia Sharmin, Labib Ahmed Leo et al. · 0 citations
Open access 2026

CroSpec-MT: A Compact Spectral State-Space Network for Joint Classification and Regression of Hyperspectral Forest Variables

Hyperspectral imagery enables simultaneous estimation of categorical and continuous forest variables, yet most deep learning approaches address these tasks in isolation and rely on heavy single-branch encoders. We propose CroSpec-MT, a compact multitask network that learns from hyperspectral patches via three parallel,...

B. Amelsha, Akshara Preethy Byju · 0 citations

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.