Skip to content

MorphAtt: A Neuromorphic Accelerator for Efficient Multi-Head Attention Processing in Spiking Vision Transformers

Sep 2026 · arXiv.org · Vol abs/2609.33207 · 0 citations · 27 references
Computer Science

TL;DR

The proposed MorphAtt is a novel digital accelerator that expedites SViT inference through streamlined processing and offers better performance and efficiency trade-offs than state-of-the-art, thereby enabling highly energy-efficient vision-based AI systems at the edge.

Abstract

Spiking Vision Transformers (SViTs) are developed as an energy-efficient alternative to conventional ViTs for computer vision tasks at the edge. However, huge parameter counts and complex multi-head self-attention (MHSA) operations make it challenging to achieve high energy efficiency in SViT inference, especially in tightly constrained applications. To maximize efficiency gains of SViT processing, we propose MorphAtt, a novel digital accelerator that expedites SViT inference through streamlined processing. Specifically, it processes MHSA operations using cascaded hardware modules: a Spiking Query-Key-Value generator (SpikeQKV), a low-complexity Spiking Multi-Head Self-Attention engine (SpikeAtten), and Reparameterization Convolution (RepConv) modules. To mitigate traffic congestion in on-chip memory accesses and data reuse, specialized inter-module buffers are integrated within the dataflow. Under synthesis using 32nm CMOS technology, MorphAtt achieves 792-1605 GOPS of throughput, while incurring ~39-55 mW of power consumption and 1.5 mm^2 of area, which lead to 20.3-29.1 TOPS/W of energy efficiency. These results also demonstrate that our MorphAtt offers better performance and efficiency trade-offs than state-of-the-art, thereby enabling highly energy-efficient vision-based AI systems at the edge.

View source

Similar papers

Book Open access Sep 2026

Algorithm--Hardware Co-Design of Spiking Transformers for In-Memory Neuromorphic Edge Vision

The Temporal Hadamard Transformer is proposed, a spiking model co-designed with hardware for in-memory edge vision that integrates sparse algorithms with parallel hardware execution, providing a practical path toward real-time edge systems under realistic resource constraints and physical device non-idealities.

Yi-Xing Li, Wen-Hua Hu, Hao-Hui Peng et al. · 0 citations
Aug 2026

Energy-Efficient SNN Accelerator with Event Scheduling for Visual Edge Intelligence

Spiking neural networks (SNNs) are progressively utilized for edge intelligence due to their replacement of dense multiply-accumulate (MAC) operations with sparse, eventdriven dynamics that are compatible with low-power hardware. Nonetheless, converting these benefits into effective item recognition and classification...

Aman Kumar, Nagendra Kumar, Manas Bajpai et al. · 0 citations
Open access Oct 2026

Beyond binary SNNs: a hardware-algorithm co-design and architectural evaluation for graded spike networks

While Spiking Neural Networks (SNNs) offer a promising path toward energy-efficient edge intelligence, conventional Binary representations often suffer from high inference latency and discretization errors. Multi-level (graded) spike models address these limitations by increasing information density per pulse, yet they...

Wen-Fei Song, A. Castagnetti, Pierre-Emmanuel Novac et al. · 0 citations

Efficient Transformer Attention for SNNs via Hadamard Simplification

This work proposes two simplified attention mechanisms, Simplified Spiking Attention (SSA) and Ultra-Simplified Spiking Attention (USSA), which replace matrix multiplication with Hadamard products and eliminate hardware-unfriendly components such as multi-head attention and scaling.

Ting-Ting Jiang, Jiangrong Shen, Long Chen et al. · 1 citation · ⚡1
Preprint Oct 2026

FlashT2S: GPU-Accelerated Tensor-to-Spike Conversion Operators for Heterogeneous Neuromorphic LLM Backends

This work examines the data-conversion components that appear on the critical path of training and inference in neuromorphic LLMs, and presents FlashT2S (Flash Tensor-to-Spike): GPU-Accelerated Tensor-to-Spike Conversion Operators, which applies architecture-aware and data-centric optimizations to reduce conversion lat...

Rong-Yuan Guo, Hao-Tian Wang, Yu-Feng Zhang et al. · 0 citations
#edge computing Review Open access Sep 2026

GPU and RISC-V acceleration for neuromorphic computing based on spiking neural networks: taxonomy, comparison, and open challenges

This survey reviews 71 studies published between 2015 and 2025 and organizes them into a taxonomy that classifies GPU works into simulation frameworks, training-acceleration techniques, large-scale and multi-GPU simulation, and application deployments, and RISC-V works into instruction-set extensions and tightly-couple...

Edris Zaman Farsa, Amirhossein Ilkhani, Marc Reichenbach et al. · 0 citations

Related blog posts

Microsoft Research Blog Nov 16, 2026

TokenEconomizer: Learning from Experience to Make AI Coding Agents More Efficient

AI coding agents are increasingly help developers fix bugs and build features largely on their own, but every interaction with their models consumes tokens. In each session, an agent sends instructions, tool descriptions, and session history to its model. Most efforts to reduce this cost focus on individual tasks. Yet developers return to the same […] The post TokenEconomizer: Learning from Experience to Make AI Coding Agents More Efficient appeared first on Microsoft Research.

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.