Large-scale video diffusion models (V-DMs) have achieved remarkable text-to-video generation quality, yet their massive computational complexity makes deployment costly. Post-Training Quantization (PTQ) offers an appealing route to accelerate inference without retraining, but existing diffusion PTQ methods remain fragile on modern V-DMs. A key reason is that contemporary V-DMs are intrinsically multi-resolution due to multi-stage training, while most prior PTQ pipelines calibrate at a fixed resolution, causing suboptimal calibration signals and biased distributions under resolution changes. To address this gap, we propose S $^{2}$ Q-VDiT $^+$, a multi-resolution co-design PTQ framework from data, supervision, and quantizer perspectives. First, Denoising-Prior Based Multi-Resolution Sampling constructs resolution-consistent noisy latents by mapping to the clean space and re-noising, together with a trajectory-aware resolution policy across timesteps. Second, Structure-Aware Multi-Resolution Distillation enhances structural alignment via window-wise distillation and transfers resolution-aware spatial dependencies via multi-scale attention distillation. Third, Debiased Modulated Quantization mitigates skewed distributions using asymmetric weight quantization and a fuseable activation debiasing scheme. Extensive experiments on multiple state-of-the-art video generation models demonstrate that S$^{2}$ Q-VDiT$^+$ consistently outperforms strong PTQ baselines under W4A6 and W4A4, delivers up to $2.08\times$ end-to-end speedup, and reduces model storage and inference memory by up to $3.8\times$ and $2.1\times$, respectively.
Weilun Feng, Chuanguang Yang, Haotong Qin et al.· IEEE Transactions on Pattern...· 2 citations
Vision foundation models are increasingly reused as frozen backbones for downstream visual recognition, making parameter-efficient adaptation a central problem. Prompt-based adaptation, including Visual Prompt Tuning (VPT), provides a lightweight way to specialize these models, but its layer-wise behavior remains poorly understood: performance is sensitive to prompt depth, placement, and task distribution, and gains on standard in-domain benchmarks do not always translate into robust generalization. We argue that this limitation is not solely an optimization issue, but a layer-wise information allocation issue: existing prompt-based methods lack principled control over what prompt-conditioned representations should preserve, suppress, and propagate across depth. Inspired by the Information Bottleneck principle, we introduce Prompted Information Bottlenecks (PIB), a framework that regularizes layer-wise compression-sufficiency trade-offs and promotes a more coherent cross-layer information path. The key idea is that effective adaptation should be minimal yet sufficient, retaining task-relevant local evidence in earlier layers while progressively discarding nuisance factors and redundant details in deeper layers. Extensive experiments show that PIB achieves strong performance across 34 datasets, reaching 92.1% on FGVC, 93.01% on HTA, and 77.33% on VTAB-1k, while tuning only 0.35% parameters on average across the main settings. Beyond benchmark accuracy, PIB helps explain the non-monotonic behavior of prompt capacity scaling, reduces shortcut reliance, and improves robustness under distribution shift and fine-grained recognition settings. These results position PIB as both a practical method and an information-allocation perspective for adapting frozen vision foundation models. Our code is available at https://github.com/itsnotacie/MM-26-PIB
Yuqi Li, Xi Xiao, Yunbei Zhang et al.· arXiv.org· 4 citations
