2025
Accurate KV Cache Eviction via Anchor Direction Projection for Efficient LLM Inference
A novel method, namely AnDPro, is proposed, which introduces a projection-based scoring function to more accurately measure token importance and guide more accurate token selection in key-Value cache eviction.
Zijie Geng, Jie Wang, Ziqi Liu et al.
· Neural Information Processin... · 6 citations