Enhancing spiking transformers with temporal feedback coding and global-local dynamic neurons.

Abstract

Spiking Neural Networks (SNNs) have emerged as energy-efficient, biologically more plausible alternatives to Artificial Neural Networks (ANNs), and their recent integration with Transformer architectures has demonstrated impressive performance on vision tasks. However, most Transformer-based SNNs rely on direct coding and Leaky Integrate-and-Fire (LIF) neurons, which leads to spike pattern collapse and insufficient temporal modeling capacity, respectively. To overcome these limitations, we propose a complementary approach with two new components. First, we introduce a Temporal Feedback Coding (TFC) scheme that leverages feedback at the encoding stage to diversify spike patterns. Second, we design a Global-Local Dynamic LIF (GLD-LIF) neuron that enhances cross-step dependency modeling by integrating local aggregation and global initialization. Extensive experiments on three Transformer-based SNN backbones and five datasets across various time steps demonstrate consistent improvements. Our method achieves accuracy gains of up to 3.64% on N-Caltech101 and 1.02% on ImageNet-1K with an increase of 1.7% parameters and 5.01% additional energy consumption. Comprehensive analyses of spike pattern statistics, attention heatmaps, shuffle-time tests and corruption robustness evaluations further verify the effectiveness and broad compatibility of our approach.