Sequential Functional Structured Tucker Compression for Large Language Model Attentions
Post-training compression of LLM attention is often formulated as independent matrix approximation, ignoring both the shared structure among attention projections and the representation shift introduced by earlier compression. We propose FTC, a sequential structured compression framework that adapts the approximation t...
Jiang-Feng Chen, Xin-Yu Wang, Tian-Shuo Yan et al.
· 0 citations