LLM agents automate GPU kernel optimization by repeatedly composing candidates and measuring their duration on real GPUs. Existing systems preserve measurement fidelity by reserving a GPU for an entire agent session or benchmarking command. However, this results in poor utilization because only a small fraction of comm...
Tian-Yu Feng, Hao-Xuan Yu, Tian-Yuan Wu et al.· 0 citations
Efficient serving of long-context reasoning-and-acting (ReAct) agents relies on KV cache reuse to reduce large language model (LLM) prefill latency and monetary cost. However, a semantic gap exists between agent harnesses and the underlying serving stack. Through context mutation, tool execution, and subagent coordinat...
Kai-Hua Fu, Yu-Kun Zhou, Chaokun Chang et al.· 0 citations
ServerlessT2I is presented, a serverless-native system that decomposes a T2I workflow into loosely coupled model functions that can be independently managed and scheduled and introduces a fair scheduler for multi-tenant serving.
Xiao-Xiao Jiang, Su-Yi Li, Sheng Yao et al.· arXiv.org· 0 citations
This work presents AgentSysBench, a benchmark suite and measurement toolkit with ten representative agentic applications and unified systems-level instrumentation, and identifies six properties that distinguish agentic workloads from conventional LLM serving.
Chaokun Chang, Yu-Kun Zhou, Kai-Hua Fu et al.· 11 citations
Gleam is a novel and network-efficient framework for task-generic GPU sharing across local-area CUDA devices, with three key contributions that reduce bandwidth overhead in CUDA API remoting through automatic model weight caching, and mitigate accumulated latency from frequent API calls by asynchronous execution.
