Aug 2026· Neural Computation· pp.
1-22
· 0 citations
Medicine
TL;DR
It is demonstrated that TNA significantly enhances classification performance across various data sets and can be applied for reducing SNNs to ternary weight precision for inference, and suggests further exploration into the application of TNA on different network architectures and data sets.
Abstract
The proliferation of artificial neural networks (ANNs) has led to increased energy consumption, raising concerns about their sustainability. Spiking neural networks (SNNs) operate using sparse, binary spikes to communicate information between neurons and offer a potential solution due to their limited energy requirements. Another technique for reducing a neural network's footprint is quantization, which compresses weight representations to decrease memory usage and energy consumption. In this study, we present twin network augmentation (TNA), a novel training method aimed at improving the performance of SNNs on a range of benchmark data sets while also facilitating enhanced network compression through quantization of weights. TNA involves cotraining an SNN with a twin SNN with identical network architecture, optimizing both networks to minimize their cross-entropy losses and the mean squared error between their output logits. We demonstrate that TNA significantly enhances classification performance across various data sets and can be applied for reducing SNNs to ternary weight precision for inference. Our results show that TNA outperforms traditional knowledge distillation methods and achieves state-of-the-art performance for the evaluated network architecture on benchmark data sets, including CIFAR-10, CIFAR-100, and CIFAR-10-DVS. This letter underscores the effectiveness of TNA in bridging the performance gap between SNNs and ANNs and suggests further exploration into the application of TNA on different network architectures and data sets.
This work analyzes flaws of conventional conversion pipelines from residual membrane potential statistics and proposes a novel conversion strategy combining dynamic initial potential tuning and feature enhancement, which generalizes to ReLU CNNs, ANN Transformers, and multi-threshold SNN variants.
Zirui Chen, Zihan Huang, Tong Bu et al.· 0 citations
A Spiking Neural Network (SNN) is a kind of brain-inspired and event-driven network, which is becoming a promising energy-efficient alternative to Artificial Neural Networks (ANNs). In recent years, SNN methods have been successfully applied in the fields of electromagnetic signal processing and image signal processing, particularly in application scenarios that require low energy consumption. However, the performance of SNNs by direct training is far from satisfactory. In this paper, we study a novel learning method named SAD-SNN (Spatial-Activation Distillation for Spiking Neural Networks), which utilizes the ANN model to guide the SNN model learning. Unlike prior works that rely on element-wise feature alignment approaches, SAD-SNN aligns spatial-activation maps at different resolutions of the teacher and student networks. Specifically, we introduce a direct alignment approach, which defines a spatial-activation loss and normalizes the representation vectors of ANN and SNN, to alleviate the unexpected precision loss. This enables the knowledge of teacher ANNs to be effectively transferred to train student SNNs. On three image classification datasets, our proposed SAD-SNN outperforms other SNN training methods no matter whether homogeneous or heterogeneous teacher ANNs are used. Furthermore, we apply SAD-SNN to the electromagnetic signal detection task, demonstrating strong generalization ability and superior performance. In conclusion, the experimental results on various tasks and SNN architectures demonstrate that our method is a general and effective solution that significantly improves the learning of student SNNs with only two time steps.
Chongxiao Qu, Qian Zhang, Chenxiao Dou et al.· Italian National Conference...· 0 citations
It is aimed at proving that SNNs have potential in such areas as computer vision, robotics, and speech recognition, and their role in overcoming the barrier between artificial and biological neural systems is proved.
Mesala Sravani, K. Kumari, S. M. Reddy· International Journal of Unc...· 0 citations
Extensive experiments on static and neuromorphic benchmarks show that lower-bit BASC models match or outperform higher-bit baselines and retain this accuracy advantage after structured pruning, while further reducing model storage and synaptic operations.
Linliang Chen, Yan Zhong, Xin Liu et al.· 0 citations
This thesis investigates stable learning and compute-resource efficiency on spiking neural networks and hybrid classical-quantum neural networks, which have become more complex architectures such as spiking neural networks and quantum neural networks.
Spiking neural networks (SNNs) offer an energy-efficient alternative to conventional deep neural networks by exploiting sparse event-driven computation, but their training remains challenging because the non-differentiable spike function requires surrogate gradients whose fixed shape may be suboptimal across layers and training stages. In this work, we introduce SAGE, an uncertainty-modulated surrogate-gradient mechanism for Transformer-based SNNs. SAGE estimates block-level uncertainty from normalized self-attention entropy and uses this signal to adapt the surrogate-gradient slope during training while leaving the inference model unchanged. By modulating only the training-time surrogate parameter, the proposed method preserves the original architecture and deployment cost while improving optimization flexibility. Experiments on CIFAR-10/100 demonstrate that SAGE achieves improved accuracy over fixed-surrogate baselines, with results up to 1-2\% consistent gains across multiple simulation time steps. These results highlight the potential of attention-derived uncertainty as a lightweight training signal for adaptive surrogate-gradient learning in transformer-based SNNs.
K. Nair, Rodrigue Rizk, K. Santosh· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.
