CRAFT achieves competitive performance with existing methods while requiring only extremely low Tucker adaptation parameters, and its efficiency advantage grows with model scale, as the frozen Tucker factors better capture the richer cross-layer structure of larger pre-trained models.
Kasun Dewage, Marianna Pensky, Suranadi De Silva et al.· arXiv.org· 1 citation
Many post-training quantization (PTQ) methods use layer-wise reconstruction, second-order proxy objectives, or activation-aware transformations to reduce quantization-induced error. Whether that error signal predicts the downstream functional impact of quantizing an individual attention projection has not been directly...
Kasun Dewage, Marianna Pensky, Suranadi De Silva· 0 citations
The results show that weight-only statistical scoring can match or outperform data-dependent methods for structured head pruning, providing a practical, zero-cost criterion for Transformer compression.
Kasun Dewage, Marianna Pensky, Heranga K. Rathnasekara et al.· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.