LT-OPD, a training framework for extreme visual-token reduction, is proposed and it is shown that on-policy learning can substantially recover capabilities lost to extreme visual-token reduction.
JustQuant is proposed, a simple yet effective framework that moves the complexity of low-bit quantization from deployment-time operators into the training process and introduces Theseus QAD, a quantization-aware distillation method that progressively applies multi-level supervision, analogous to the gradual replacement...
Kai-Cheng Yang, Kai-Sen Yang, Chun-Yu Liu et al.· arXiv.org· 0 citations
FOCUS is proposed, a post-training quantization framework with end-to-end scale learning for FP4 Optimization via Coupled-Relaxation and Dual-Granularity Scaling, which relaxes the tight coupling between quantization and dequantization scales with a learnable full-precision coefficient, enabling more effective optimiza...
Xiang-Long Yan, Hong Liu, Cheng-Zhu Bao et al.· 1 citation
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.