Fei Fang

Tessera: Demand-Driven KV Cache Management for Retrieval-Augmented LLM Serving

RAG and retrieval-based agent memory both inject retrieved content into LLM prompts, as document chunks and recalled memory records, respectively. The same content can recur across requests at different prompt positions or after different preceding contexts, preventing reuse through conventional prefix caching. Our cha...