KV Admission: Learning What to Write for Efficient Long-Context LLM Inference
This paper formalizes KV management as a causal system of three primitives: KV Admission, Selection, and Eviction, and instantiate KV Admission via Write-Gated KV (WG-KV), a lightweight mechanism that learns to predict token utility before cache entry.
Yen-Chieh Huang, Rui Fang, Ming-Syan Chen et al.
· 2 citations