Kalpit Patel

Scaling Long-Context LLMs via Unified KV Cache Optimization: A Comparative Study of Paged Attention and Quantization

The Key-Value (KV) cache is the dominant memory bottleneck in autoregressive large language model (LLM) inference, growing linearly with context length. This paper presents a systematic empirical study of three KV cache optimization strategies applied to Mistral-7B-Instruct-v0.2 on a single NVIDIA A100 GPU: (1) a full-...