Virtual Head Attention
The design of attention mechanisms in long-context large language models faces an ``impossible triangle''among KV cache size, training and prefill FLOPs, and model quality. GQA achieves strong quality but incurs high KV cache overhead; alternative approaches such as MLA and MFA can compress the cache, but MLA suffers f...
Xi-Bo Ding, Guo-Xia Wang, Jin-Le Zeng et al.
· 0 citations