Preprint
Oct 2026
SPECMEM: Accelerating Agentic Tool Calling via Live Memory Management
Tool-calling agents are mostly optimized for accuracy; in deployment, however, their growing cost is latency: every tool call is decoded autoregressively. Speculative decoding accelerates this without changing the output, and retrieved past calls make strong drafts under real use, where each user's behavior tends to re...
Zi-He Song, Wen-Ting Zhao, Ye Liu et al.
· 0 citations