Agents tend to optimize, select, or constrain execution structures before decisive runtime outcomes are observed. However, such pre-execution commitment creates an orchestration bottleneck: when intermediate evidence invalidates the pending continuation, agents must either execute stale steps or replan broadly, compoun...
Tian-Xing Wang, Ming-Ming Zhao, Shuai Huang et al.· 1 citation
C2KV is proposed, a unified framework for non-prefix KV reuse that jointly optimizes KV cache compression and concatenation that significantly reduces KV cache storage and transfer costs.
Chuheng Du, Jun-Yi Chen, Hanlin Tang et al.· Proceedings of the 32nd ACM...· 2 citations
Gleam is a novel and network-efficient framework for task-generic GPU sharing across local-area CUDA devices, with three key contributions that reduce bandwidth overhead in CUDA API remoting through automatic model weight caching, and mitigate accumulated latency from frequent API calls by asynchronous execution.