Jul 2026· International Conference on Signal Processing and Communications· pp. 1-5· 0 citations· 15 references
Abstract
Diffusion models achieve strong performance in generative modeling but incur high memory and computational costs due to large model parameters and iterative denoising procedures, limiting deployment in resource-constrained environments. This paper presents a unified framework to improving inference latency and memory efficiency of diffusion models by jointly optimizing activation reuse and weight storage. We introduce a representation-aware caching strategy(ReCaT) which reduces redundant computation during inference by selectively reusing intermediate activations along the diffusion trajectory. ReCaT identifies effective caching points based on representational similarity measured using centered kernel alignment(CKA), enabling informed activation reuse without degrading generative quality. To complement activation reuse, we propose Adaptive Clusteringguided Quantization with Entropy Encoding (ACQ-E) for efficient weight storage. ACQ-E uses histogram-guided clustering to model weights, followed by adaptive mixed-precision quantization and entropy encoding to reduce storage and cost. The proposed framework integrates representation-aware caching and adaptive quantization into a unified optimization pipeline, leading to reduced effective bit-width, smaller model size, and lower computational cost while maintaining generative fidelity. Experiments on standard benchmarks, including LSUN Churches, LSUN Bedrooms, CelebA-HQ, MS-COCO, and PartiPrompt, demonstrate efficiency gains with minimal impact on image quality when compared to full-precision.
Diffusion-based visual generative models deliver strong image and video synthesis quality but incur high inference costs because sequential samplers repeatedly evaluate large networks. Caching-based methods reduce inference latency by reusing intermediate computations across adjacent timesteps. However, existing cache controllers rely primarily on local temporal variation and overlook the trajectory-level consequences of cache reuse. We introduce Error-Propagation-Aware Cache (EpaCache), a training-free caching policy that adaptively allocates the reuse budget on timesteps with lower downstream impact. Experiments on image and video synthesis models demonstrate that EpaCache consistently improves the latency--fidelity trade-off over existing caching methods. On FLUX.1-dev, EpaCache outperforms the prior state-of-the-art caching method in both latency and fidelity, reducing inference time from $11.7$ s to $11.3$ s while improving PSNR from $21.4$ to $22.8$. On HunyuanVideo, EpaCache achieves a $2.63\times$ speedup over uncached inference and improves SSIM from $0.891$ to $0.905$ over the prior state-of-the-art method at matched latency.
Yu-Han Liu, Zong-Wei Hong, Jinglun Li et al.· 0 citations
OmniCache is proposed, a unified hierarchical caching framework that performs multidimensional feature reuse through Token Cache, Frame Cache, Block Cache, and Layered Cache that reuses spatial features in temporal layers and temporal features in spatial layers, while Layered Cache captures cross-step redundancy at the model-layer level.
Zhaoyuan He, Muhammad Muaz, Lili Qiu· arXiv.org· 0 citations
The exceptional generative capabilities of modern diffusion models are fundamentally bottlenecked by the quadratic computational complexity of their attention mechanisms. While recent feature caching strategies attempt to accelerate inference by skipping layers at static intervals, they fail to account for the non-linear evolution of latent features, inevitably causing severe structural distortions and temporal flickering. To address this, we propose AST-ToMe (Adaptive Step-Aware Thresholding), a novel dynamic gating mechanism that utilizes a runtime L2 norm feature drift metric to adaptively determine whether to compute or reuse attention states. Furthermore, we extend AST-ToMe to video stream generation through a cross-frame state inheritance design. Experimental results demonstrate that for single-image synthesis, AST-ToMe achieves a 15.3% reduction in inference latency with near-lossless perceptual quality (LPIPS: 0.0023). In continuous video generation, our method not only accelerates inference but also serves as a robust temporal anchor. By effectively suppressing random stochastic variations, AST-ToMe successfully reduces Temporal Jitter from 0.2059 to 0.1865, paving a highly efficient path for stable, flicker-free video generation.
Shu-Zhi Zheng· International Conference on...· 0 citations
LinCa decomposes cached features into sub-components with distinct continuity properties via a lightweight invertible network and applies differentiated prediction orders matched to each component, forming a unified Decompose-Predict-Reconstruct pipeline.
Jinshan Liu, Haoran Qin, Xiaobing Tu et al.· 0 citations
LaCache is proposed, a training-free acceleration framework that alleviates operator-level redundancy through lossless caching and mixed precision, and inegrates a per-group FP8 quantization strategy for FFN layers, tailored to step-dependent activation distributions across the diffusion process.
Xingru Chen, Zelang Liang, Yongjia Ma et al.· arXiv.org· 0 citations
Diffusion models have achieved dominant performance in visual generation but suffer from substantial inference overhead. While cache-based acceleration has emerged as a promising solution, existing policies rely on local similarity heuristics, which we identify as being significantly misaligned with final generation quality. This discrepancy stems from the non-uniform propagation and accumulation of errors along the denoising trajectory. To address this, we propose Global-Impact Cache (GCache). We first establish a rigorous theoretical characterization of the error propagation upper bound. Recognizing that this bound can be overly conservative for complex, highly non-convex diffusion models, we further reparameterize the propagation exponent with a Bernstein form and reformulate cache policy search as a bilevel optimization problem. In detail, GCache identifies an optimal reuse policy in the inner objective while aligning the error-weighting function with generation quality loss in the outer objective. This framework effectively reconciles theoretical rigor with empirical performance, learning to prioritize computation where it most impacts visual fidelity. Extensive experiments demonstrate that GCache consistently outperforms prior caching strategies on both video and image generation. Notably, on the state-of-the-art Wan2.1 video diffusion model, GCache maintains a 2.17x speedup while significantly enhancing generation quality, reducing LPIPS from 0.1095 to 0.0316.
Xichen Ye, Yifan Wu, Zhikang Xie et al.· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.
