A unified latency decomposition framework for dLLMs is introduced to disentangle factors and analyze their impact on inference speed in real deployments, and categorize acceleration techniques along three axes covering algorithmic innovations, architectural and system optimizations, and inference-time scaling.
Abstract
Diffusion large language models (dLLMs) offer a theoretical advantage in parallel generation over standard autoregressive models. However, parallel generation alone does not guarantee practical speedups. Realizing this efficiency requires specialized inference mechanisms, such as diffusion-aware caching and reuse. Consequently, as inference efficiency becomes a prerequisite for practical deployment, recent research has actively explored acceleration techniques across algorithms, architectures, and systems. However, rigorous comparisons remain difficult, as end-to-end latency stems from intricate trade-offs between algorithmic, architectural, and system-level factors that are often conflated in existing benchmarks. In this survey, we introduce a unified latency decomposition framework for dLLMs to disentangle these factors and analyze their impact on inference speed in real deployments. Guided by this framework, we categorize acceleration techniques along three axes covering algorithmic innovations, architectural and system optimizations, and inference-time scaling. Finally, we provide guidelines for reproducible benchmarking and highlight open challenges for realizing the full potential of parallel generation.
LaCache is proposed, a training-free acceleration framework that alleviates operator-level redundancy through lossless caching and mixed precision, and inegrates a per-group FP8 quantization strategy for FFN layers, tailored to step-dependent activation distributions across the diffusion process.
Xingru Chen, Zelang Liang, Yongjia Ma et al.· arXiv.org· 0 citations
This work proposes CAI-DLLM, a training-free inference method that uses first-step confidence to guide denoising and reduce inference time, and evaluates CAI-DLLM on LLaDA-8B-Instruct and Dream-7B-Instruct across math, code, reasoning, commonsense, and long-context tasks.
Farhana Amin, Sabiha Afroz, D. Nikolopoulos· 0 citations
With the rapid advancement of deep learning technology, the parameter scale of large language models has grown exponentially, expanding from hundreds of millions in the early stages to hundreds of billions or even trillions today. Pipeline inference is a crucial approach enabling efficient inference in large language models. However, existing pipeline inference relies on static layer allocation, ignoring the intrinsic variance in layer-wise computation and memory footprints, as well as runtime fluctuations in request rates and sequence lengths. Consequently, under dynamic workloads, compute-dense stages rapidly bottleneck the pipeline and induce severe queue blocking while leaving other devices idle, ultimately degrading end-to-end latency and severe GPU underutilization. To address these challenges, this paper proposes a dynamic pipeline parallel inference algorithm. Centering on the three phases of LLM pipeline inference—partitioning, updating, and migration—the algorithm introduces: (1) A proactive update trigger mechanism driven by multidimensional load forecasting. Rather than relying on reactive bottleneck indicators, it translates projected request rates and token lengths into stage-level VRAM demands, preemptively initiating reconfiguration only when impending hardware capacity violations are detected; (2) A joint partition-migration optimization strategy utilizing a two-stage biased random key genetic algorithm. By embedding a maximum-weight bipartite matching formulation into the evolutionary fitness evaluation, this strategy mathematically couples pipeline boundary search with physical state mapping, maximizing resident parameter reuse to guarantee minimal-overhead model migration; (3) Distributed cluster experiments conducted using public datasets and the Ray framework demonstrate that the proposed method outperforms existing state-of-the-art pipeline inference solutions in metrics including response latency and resource overhead, specifically improving throughput by 2.3% compared to the SOTA framework.
Influence functions provide a principled framework for tracing model predictions back to training data, yet existing methods remain impractical for large language models due to prohibitive storage and I/O costs. Prior approaches face two critical barriers: the Fidelity Gap from coarse curvature approximations, and the I/O bottleneck from materializing per-example gradients, which renders million-scale datasets infeasible. We propose StructInf, a structure-aware influence estimation framework that addresses both gaps through system-algorithm co-design. For fidelity, we first replace the indefinite Hessian with an adaptive block-diagonal damped empirical Fisher surrogate, computed via quadratically convergent Newton--Schulz iterations. For efficiency, we then introduce a streaming algorithm that eliminates per-example gradient storage entirely, reducing space complexity from O(N • d) to O(d) and enabling processing of million-scale training instances on consumer hardware, where N denotes the number of training instances and d the parameter size. We further identify and resolve Split-View Bias, a distributed pitfall where naive averaging of local curvatures fails, via a distributed gradient calibration strategy that synchronizes global statistics with minimal communication. Finally, we conduct comprehensive experiments to evaluate StructInf against other baselines. Unlike prior methods requiring hours of pre-computation or TB-scale storage, StructInf enables real-time data selection during training. Across GLUE and instruction-tuning benchmarks, StructInf achieves up to +26% AUC on MRPC and average 7× speedup for 7B-scale models, making high-fidelity influence analysis practical on a single RTX 4090. These results suggest that practical IF for LoRA-tuned LLMs benefits from joint optimization of curvature modeling, memory management, and distributed coordination.
Mengyi Yan, Yaoshu Wang, Guangyi Zhang et al.· Proceedings of the 32nd ACM...· 0 citations
Dynamo-MoE, an out-of-box MoE inference framework to bridge the performance gap by dynamic parallelization strategies, integrates a novel load balancing approach based on token sorting and on-demand expert loading to solve the workload imbalance issue in the scenario of high workload.
Jiahao Chen, Shigang Li, Rongtian Fu et al.· IEEE International Symposium...· 0 citations
RPS is proposed, a novel training-free decoding method that seeks mid-entropy positions as promising candidate pivots (where to decode), and determines their token assignment that yields the greatest downstream benefit via lookahead evaluation (what to decode).
Yushi Ye, Xu Chen, Hao-Yun Jiang et al.· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.
