Kwai Summary Attention V2 Technical Report
The KSA V2 is developed, which not only achieves a more flexible KV cache memory compression but also reduces the computation cost in Full attention layer and finds that the summary tokens can serve two roles: the KV memory and sparse indexer.