Taming Dynamic Diffusion LLM Inference through Virtual Static Execution

Abstract

Diffusion-based Large Language Models (DLLMs) promise high-efficiency text generation by refining all tokens in parallel via dense matrix operations. However, this non-autoregressive paradigm introduces a significant challenge: workload dynamism. The number of active tokens fluctuates unpredictably at every denoising step, creating a fundamental conflict with modern deep learning compilers that rely on static tensor shapes for optimization. Consequently, existing serving systems face a dilemma: either incur severe latency spikes due to frequent re-compilation or suffer from low GPU utilization caused by excessive padding. We present Prism, a serving system that reconciles this conflict through a novel paradigm called Virtual Static Execution. Prism abstracts the highly dynamic diffusion workload into a predictable, static execution flow transparent to the compiler. We achieve this via three techniques: (1) Spatial Regularization: We discretize the dynamic token space into a compact set of fixed Physical Buckets, maximizing kernel reuse while minimizing compilation overhead. (2) Temporal Stabilization: We decouple logical diffusion steps from physical execution using a budget-aware scheduler, smoothing out bursty workloads and enabling computation-communication overlap. (3) Compute Optimization: We introduce specialized kernels that selectively bypass padding data, ensuring that the regularization overhead does not translate into wasted compute. Evaluated on LLaDA-8B and Mixture-of-Experts (MoE) models, Prism outperforms state-of-the-art baselines by 5.8 × in throughput and reduces median latency by 70%.