Skip to content

TriM-D: Accelerating 2-D/3-D Diffusion Generation Via Triple-Mixed Quantization and Sparsity-Aware Dual-Branched Computation

Sep 2026 · IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems · Vol 45, pp. 4380-4393 · 0 citations · 40 references

Abstract

Diffusion models have shown marked advancements in 2-D generation and have also become focal points in 3-D generation via consistent multiview image generation. However, the computation and memory demands hinder their real-time deployment on mobile and edge devices. Moreover, the reduction of diffusion timesteps leads to a decrease in interstep similarity, thereby making previous methods ineffective in optimizing computation. The varied layer quantization sensitivity and optimal format in diffusion models also present challenges for traditional quantization methods to achieve efficient memory compression. To address these issues, we first introduce <inline-formula> <tex-math notation="LaTeX">${M}^{3}$ </tex-math></inline-formula> quantization, a mixed-precision, mixed-format, and mixed-granularity quantization framework that allocates optimal precision and format to different data with a unified FP8 computation flow for efficient implementation. Building upon <inline-formula> <tex-math notation="LaTeX">${M}^{3}$ </tex-math></inline-formula> quantization, we present TriM-D, a hardware accelerator designed to optimize computation. It features sparsity-aware dual-branched FP8-MAC units that introduce a dual-branch path and an efficient approximate adder to fully utilize bit-level sparsity of <inline-formula> <tex-math notation="LaTeX">${M}^{3}$ </tex-math></inline-formula>-quantized data. Such an FP8-MAC scheme preserves both efficiency and numerical accuracy. Our experiments demonstrate that TriM-D significantly outperforms the state-of-the-art (SOTA) diffusion accelerator Cambricon-D (Camb-D), achieving an average reduction of 56.5% in memory access and 80.6% in computation cost. In addition, TriM-D provides a <inline-formula> <tex-math notation="LaTeX">$55\times $ </tex-math></inline-formula>, <inline-formula> <tex-math notation="LaTeX">$6.72\times $ </tex-math></inline-formula> improvement in energy efficiency and <inline-formula> <tex-math notation="LaTeX">$3.69\times $ </tex-math></inline-formula>, <inline-formula> <tex-math notation="LaTeX">$2.67\times $ </tex-math></inline-formula> speedups over NVIDIA A100 and Camb-D.

View source

Similar papers

Aug 2026

S $^{2}$ Q-VDiT$^+$: Accurate Quantized Video Diffusion Transformer with Multi-Resolution Sampling and Structural Distillation.

Large-scale video diffusion models (V-DMs) have achieved remarkable text-to-video generation quality, yet their massive computational complexity makes deployment costly. Post-Training Quantization (PTQ) offers an appealing route to accelerate inference without retraining, but existing diffusion PTQ methods remain fragile on modern V-DMs. A key reason is that contemporary V-DMs are intrinsically multi-resolution due to multi-stage training, while most prior PTQ pipelines calibrate at a fixed resolution, causing suboptimal calibration signals and biased distributions under resolution changes. To address this gap, we propose S $^{2}$ Q-VDiT $^+$, a multi-resolution co-design PTQ framework from data, supervision, and quantizer perspectives. First, Denoising-Prior Based Multi-Resolution Sampling constructs resolution-consistent noisy latents by mapping to the clean space and re-noising, together with a trajectory-aware resolution policy across timesteps. Second, Structure-Aware Multi-Resolution Distillation enhances structural alignment via window-wise distillation and transfers resolution-aware spatial dependencies via multi-scale attention distillation. Third, Debiased Modulated Quantization mitigates skewed distributions using asymmetric weight quantization and a fuseable activation debiasing scheme. Extensive experiments on multiple state-of-the-art video generation models demonstrate that S$^{2}$ Q-VDiT$^+$ consistently outperforms strong PTQ baselines under W4A6 and W4A4, delivers up to $2.08\times$ end-to-end speedup, and reduces model storage and inference memory by up to $3.8\times$ and $2.1\times$, respectively.

Weilun Feng, Chuanguang Yang, Haotong Qin et al. · 2 citations
Preprint Aug 2026

APT: Accelerating Diffusion Transformers via Attention Probability-Guided Pruning and Quantization

APT, a software-hardware co-designed accelerator for high-resolution DiTs that leverages attention probabilities as a unified importance metric to jointly optimize computation through fine-grained pruning and adaptive precision scaling, and is evaluated on SOTA DiT models including PixArt, Stable Diffusion 3, and FLUX.

Sungyeob Yoo, Seeyeon Kim, Joonyong Park et al. · 0 citations
Preprint Aug 2026

Fixed-Budget Gaussian Volume Encoding with Structure-Aware Allocation

Scientific simulations often produce scalar volumes faster than they can be stored, transferred, and loaded, while in situ reduction must use only a limited share of simulation resources. This work encodes scalar fields as anisotropic Gaussian primitives under a fixed budget. The complete primitive set is allocated analytically from local field structure, including position, orientation, and shape, then refined directly against the scalar field without densification, pruning, or count changes. The selected budget determines encoded storage before refinement and, together with the iteration schedule, provides a controllable refinement-time budget. In a controlled benchmark, truncation-aware field evaluation reduces encoding time by up to 51x; 1.4 million Gaussians encode a billion-voxel volume in at most four minutes on one desktop GPU, with reduced-iteration refinement completing in under one minute. Across five datasets spanning 2.1 million to 1.1 billion evaluated voxels, compression-useful configurations achieve 15.0-38.7 dB PSNR at compression ratios from 2.2x to over 40,000x. Pre-encoding structure statistics characterize fields for which one-shot allocation yields limited gains from additional capacity. Because primitives retain scalar attributes rather than baked appearance, a single compact model serves every subsequent visualization state - supporting post-hoc transfer-function, colormap, lighting, and viewpoint changes without re-encoding.

Michael R. Martin, J. Insley, Victor A. Mateevitsi et al. · 0 citations
Preprint Aug 2026

GeoCache: Training-Free Acceleration of Multi-View Texture Diffusion via Geometric Delta Transport

Geometry-conditioned multi-view diffusion enables high-quality 3D texture generation, but its repeated per-view denoiser evaluations introduce substantial computational cost. Existing training-free accelerators primarily exploit temporal redundancy by reusing computation across denoising steps. In multi-view texturing, however, skipping a step also removes the cross-view interaction that continually aligns different observations of the same surface, leading to rapidly degraded consistency and fidelity. Our analysis identifies a complementary source of redundancy: although intermediate features remain view-specific, geometrically corresponding surface points exhibit transferable evolution in their predicted clean signals. Based on this observation, we introduce \gc{}, a training-free plugin that evaluates a rotating subset of anchor views and transports their geometry-aligned per-step $\xz$ updates to the remaining views. Periodic full-view computation controls accumulated error, while sampler-consistent reconstruction preserves the denoising trajectory. \gc{} requires neither retraining nor architectural modification and uses the position maps already available in geometry-conditioned texturing pipelines. Across Hunyuan3D-2.1, SyncMVD, and MVPainter, \gc{} achieves a stronger speed--fidelity trade-off than temporal caches and step reduction at operating points above $2\times$. On Hunyuan3D-2.1, it delivers a $2.21\times$ denoiser-loop speedup with an MV-LPIPS of 0.0293 and an MV-PSNR of 33.60 dB, providing the best fidelity among all tested methods above $2\times$. The same transferred configuration reaches the highest speedup and lowest FLOPs on SyncMVD, while \gc{} achieves the lowest FLOPs and best fidelity among the accelerated methods on MVPainter. These results establish cross-view geometry as an effective acceleration axis for multi-view texture diffusion.

Haotang Li, Zhenyu Qi, Shaohan Wang et al. · 0 citations

Related blog posts

MIT News · Artificial Intelligence Sep 14, 2026

New method enables AI for safety-critical situations

The “HardFlow” algorithm could help generative AI models produce high-quality outputs that obey strict requirements when “pretty close” doesn’t cut it.

GPT-Lab Sep 10, 2026

Responsible AI Must Consider Its Afterlife

AI may appear weightless, but every model depends on physical infrastructure. To understand responsible AI, we need to look beyond algorithms and consider the entire lifecycle of the hardware behind them. The post Responsible AI Must Consider Its Afterlife appeared first on GPT-Lab.

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.