Conference
Open access
2026
SpecCache: Speculative KV Cache Reuse for Efficient RAG Serving
This work finds that the deep-layer features of a lightweight speculative model exhibit strong consistency with the target model in the selection of critical tokens for recomputation, and proposes SpecCache, which employs deep-layer hidden-state norms from a speculative model as a proxy to guide the critical token selection for target large model.
Zijian Wen, Tao Zhang, Shuangwu Chen et al.
· Annual Meeting of the Associ... · 1 citation