Xi-Bo Ding

Virtual Head Attention

The design of attention mechanisms in long-context large language models faces an ``impossible triangle''among KV cache size, training and prefill FLOPs, and model quality. GQA achieves strong quality but incurs high KV cache overhead; alternative approaches such as MLA and MFA can compress the cache, but MLA suffers f...