ME-MoD: Algorithm-Hardware-Dataflow Co-Design for Memory Efficient Mixture-of-Depth-Based Vision Transformer Accelerator

A High-Performance and Low-Resource Reconfigurable Hardware Implementation for Softmax in Transformer

The efficient implementation of the softmax is critical for optimizing transformer hardware accelerators. Unlike its role as a static, one-time classifier in convolutional neural networks (CNNs), softmax in transformers is core to achieving dynamic contextual awareness, generating attention weights that enable the model to focus on relevant information. Its high-frequency, large-scale execution across every layer makes the softmax a critical performance bottleneck, particularly when processing long sequences. Existing hardware implementations face two primary challenges: excessive resource consumption from the exponential (EXP) computation, and high latency induced by the softmax’s nonelementwise nature. This article proposes a domain-unified piecewise EXP approximation technique. This technique reduces the resource consumption of the EXP unit by 23% and its approximation error by 26%. Furthermore, the proposed design provides direct and efficient hardware-level support for the online softmax algorithm, significantly reducing latency. Under the TSMC 40-nm process, it achieves a $1.36\times $ improvement in area-normalized throughput. On a Xilinx KCU116 FPGA, LUT-normalized throughput improves by $1.26\times $ . Moreover, the proposed reconfigurable architecture substantially improves resource utilization. Finally, end-to-end evaluations confirm that the design strictly preserves native inference accuracy while delivering a notable 4.86% overall system-level latency reduction for Llama3-8B.

Analysis of Research Progress on Deployment Methods for Deep Learning Models on FPGAs

Deep learning (DL) models have achieved remarkable progress in natural language processing, computer vision, content generation, and edge intelligence; however, their rapidly increasing computational complexity, memory demand, and deployment diversity pose significant challenges for practical implementation. Field-programmable gate arrays (FPGAs) provide customized low-precision computation, spatial dataflow, on-chip data reuse, reconfigurability, and rich I/O capabilities, making them an important platform for DL inference. This paper presents a systematic review of FPGA-based DL deployment from a cross-layer perspective spanning model, compiler, architecture, runtime, and electronic design automation (EDA). Following a PRISMA-guided evidence synthesis protocol, this review analyzes DL workload characteristics, FPGA architectural optimizations, deployment toolflows, and physical implementation challenges. A unified taxonomy is proposed along the specialization–programmability continuum, including model-fixed accelerators, generator-based accelerators, template-configurable accelerators, and ISA-programmable overlays. These approaches are compared according to hardware regeneration requirements, model adaptability, operator coverage, compilation cost, and deployment flexibility. Furthermore, emerging workloads, including vision Transformers, graph neural networks, large language models, and multimodal models, are analyzed from the perspectives of computation, memory behavior, and runtime coordination. The review shows that FPGA deployment efficiency increasingly depends on memory capacity, mutable state management, operator support, and end-to-end compilation capability rather than peak multiply–accumulate throughput alone. Based on the analysis of 70 primary FPGA implementation studies, this paper highlights that reliable cross-study comparison requires careful consideration of model configuration, precision, execution phase, batch size, memory residency, FPGA platform, and evidence maturity. For multimodal generative models, the current evidence remains limited, with no identified end-to-end FPGA-based vision–language model implementation in the reviewed corpus. This review provides a systematic perspective for future FPGA-based DL deployment research, emphasizing cross-layer optimization, physically aware compilation, extensible accelerator architectures, and practical deployment efficiency.

A Memory-Efficient Depthwise Separable Convolution Accelerator Using Run-Length Coding

On edge devices, convolutional neural network (CNN) inference is bottlenecked mainly by memory bandwidth, owing to the frequent memory accesses to feature maps and parameters. To address this challenge, we propose a memory-efficient hardware accelerator for depthwise separable convolution that minimizes off-chip memory traffic and parameter storage. The proposed architecture employs three key techniques: (1) a 64-bit run-length coding (RLC) packet compression that exploits feature-map sparsity after ReLU, (2) a mixed-precision scheme that represents feature maps and weights at different precisions, and (3) separated depthwise and pointwise convolution units. In particular, feature maps are transferred in RLC-compressed form, which reduces the amount of data exchanged with the host. The compressed data are decoded row by row, so the on-chip buffers hold only the rows required for computation rather than a complete feature map. In software simulation on ImageNet, the mixed-precision scheme reduced the parameter storage by 49.22% at a cost of 6.24 percentage points (pp) in Top-1 accuracy, and the RLC reduced the data by up to 56.07% in the deeper layers. Implemented on a Xilinx ZCU-104 FPGA, the proposed accelerator performs the depthwise separable convolution with a small number of logic resources and on-chip memory, confirming its feasibility for resource-constrained edge devices.

A Low-Power Sparse Convolution Accelerator with Idle-First-Task-Assignment for Edge Vision

In recent years, edge-vision monitoring systems for applications such as smart animal husbandry have faced strict tripartite constraints: maintaining input resolution under extremely limited transmission bandwidth and strict power budgets. Conventional dense convolutional neural networks (CNNs) cannot satisfy the resource limits of such constrained IoT nodes. To address this challenge, this paper presents a low-power sparse convolution accelerator for edge devices, fabricated and validated in a 16 nm process. First, the accelerator adopts a bitmap-based format for compression in both data transmission and computation, effectively reducing memory and bandwidth overhead. Second, to mitigate load imbalance in sparse computation, an Idle-First-Task-Assignment (IFTA) dynamic scheduling strategy is proposed, significantly reducing processing-element (PE) idle time and improving multiplier utilization. In addition, a dedicated dataflow is designed to support and accelerate depthwise separable convolution (DWConv), which is widely used in lightweight networks. Experimental results show that the chip occupies only 0.5~mm$^2$ core area and consumes as little as 12--16~mW. On ImageNet, for sparse VGG16 and MobileNetV2, the proposed accelerator achieves 6.5$\times$ and 2.8$\times$ speedups, respectively, over traditional dense accelerators, and also delivers significant performance gains over the existing sparse accelerator.

FluxBin: Flexible LUT-based Ultra-low-bit LLM Inference by Algorithm-Kernel Synergy

FluxBin is proposed, an algorithm-kernel co-design that synergizes post-training quantization with a highly optimized CUDA kernel and introduces Decoupled Row-Column Binary Decomposition to enhance representational capacity while maintaining hardware efficiency.

CDCA: a column-serial and dual-engine collaborative accelerator for depthwise separable convolution