LLM-based multi-agent systems (LLM-MAS) use specialized agents that exchange messages to accomplish complex tasks. Their reliance on networked protocols, remote tools, and model endpoints exposes communication to message loss, delay, and context expiration. Because agents often exchange open-ended natural language, ano...
Jin-Hao Zhou, Xiao-Yue Zhang, Jiongchi Yu et al.· 0 citations
DoMoE is proposed, a domain-aware MoE inference system that exploits domain locality in inference workloads and organizes routing information into domain-specific expert routing tables, restricts semantic matching to domain-relevant tokens and explicitly balances prediction accuracy against prediction overhead.
Yao Mu, Fa-Hao Chen, Wen-Bin Zhu et al.· Proceedings of the Thirty-Fi...· 1 citation
Modern large language model (LLM) inference suffers from severe Time-To-First-Token (TTFT) bottlenecks. Existing prefix KV caching mechanisms are inherently stateless, forcing a trade-off between cross-chunk attention accuracy and online recomputation overhead. To address this issue, we propose Pegasus, a novel statefu...
Fahao Chen, Peng Li, Dongxiao Yu et al.· Fall Joint Computer Conferen...· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.