PatchKV: Efficient KV Cache Recovery for Dynamically Edited LLM Contexts
PatchKV, a profile-guided recovery system for suffix-preserving revisions, decomposes adjacent context versions into an exact prefix, an updated span, and an aligned suffix, and achieves a speedup in mean resume time-to-first-token over full suffix recomputation.
Guo-Tao Yang, Rui Guo, Si-Wei He et al.
· 0 citations