Tool-calling agents are mostly optimized for accuracy; in deployment, however, their growing cost is latency: every tool call is decoded autoregressively. Speculative decoding accelerates this without changing the output, and retrieved past calls make strong drafts under real use, where each user's behavior tends to re...
Zi-He Song, Wen-Ting Zhao, Ye Liu et al.· 0 citations
BlockServe is presented, a continuous batching framework that integrates block-grained scheduling -- immediately evicting completed requests at block boundaries -- with mixed-state execution that extends dual cache and parallel decoding to heterogeneous batches via gather-scatter indexing.
Yuanjie Zhu, Liangwei Yang, Ke Xu et al.· arXiv.org· 0 citations
This survey investigates the current research landscape of multimodality modeling from three perspectives: the first group of multimodal models adopts a heterogeneous architecture to bridge different modality data, the second leverages LLM for multimodality modeling via a unified language modeling objective, and the th...
Zhongfen Deng, Yibo Wang, Yueqing Liang et al.· 2 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.