As Large Language Models (LLMs) become essential in privacy-sensitive sectors like hospitals and government agencies, the on-premise LLM servers offer a cost-effective and secure alternative to public cloud services. However, these resource-constrained servers struggle to guarantee heterogeneous Service Level Objective...
Ze-Shen Zhang, Han Zhao, Wei-Hao Cui et al.· 0 citations
Aethon is presented, a memory offloading system for public clouds that maximizes offloaded data while ensuring each application satisfies the service-level agreement (SLA).
Guang-Qiang Luan, Pu Pang, Quan Chen et al.· Proceedings of the Internati...· 0 citations
As applications demand increasing memory capacity in clouds, memory pooling provides a cost-effective way to improve utilization and expand capacity. Compute Express Link (CXL), which enables high-performance direct access to remote memory, makes this approach increasingly practical. However, existing studies of memory...
Guangqiang Luan, Pu Pang, Quan Chen et al.· Proceedings of the Internati...· 0 citations
Prefix caching has become a key technique for LLM serving, and nowadays the reusable KVCache contents are often hosted on distributed servers. For long-context LLM inferences with high cache hit ratio, cross-server KVCache transmission has become an emerging performance bottleneck; such network-intensive LLM inferences...
Weiye Wang, Chen Chen, Junxue Zhang et al.· Asia-Pacific Workshop on Net...· 0 citations
Federated Learning (FL) allows edge clients to collaborate in model training with data privacy preserved, yet it is known to suffer low training efficiency and model accuracy. Given that efficiency and accuracy are usually conflicting objectives, existing practices increasingly employ an adaptive scheme that changes th...
Jiayi Zhang, Zuo Gan, Chen Chen et al.· IEEE Transactions on Mobile...· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.
