A 11.0-TOPS/W Diffusion Accelerator With Temporal Data Reuse for Real-Time Text-to-Motion Generation

Abstract

Text-to-motion models are AI systems that generate human motion sequences directly from natural language descriptions, serving as key enablers for immersive virtual avatars and interactive digital humans in AR/VR ecosystems. However, state-of-the-art text-to-motion diffusion models suffer from substantial computational costs due to their iterative nature, making them ill-suited for deployment on resource-constrained edge devices. To address these challenges, we propose MoDiff, a hardware–software codesigned processor that exploits temporal redundancy across denoising steps. MoDiff introduces a sparse feed-forward network (FFN) computation strategy that selectively recomputes only critical GELU activation outputs, effectively bypassing redundant operations without sacrificing generation quality. To efficiently support this irregular sparsity, MoDiff integrates a dynamic sparsity matmul engine (DSME) and a reconfigurable vector-processing engine (RVPE). Furthermore, MoDiff incorporates a difference-based softmax approximation (DSA) and a unified quantization flow to significantly lower power consumption and area overhead. MoDiff is fabricated in 14-nm CMOS technology with a die area of 7.16 mm2 and achieves a peak energy efficiency of 11.0 TOPS/W. Despite aggressive quantization, MoDiff maintains motion generation quality comparable to floating-point inference, establishing a compelling solution for real-time generative AI on edge platforms.