Mixture-of-Experts (MoE) inference on memory-constrained edge platforms is often dominated by expert-weight movement rather than computation. Although each token activates only a small subset of experts, routing is input-dependent, so all experts must remain accessible and memory movement dominates latency. Existing ap...
Cong-Wei Zhang, Youming Tao, Yi-Fei Zou et al.· Proceedings of the 32nd Annu...· 1 citation
An SFC Deployment and Scheduling strategy based on the Soft Actor-Critic (SAC) algorithm, named SAC-DS, which integrates SFC deployment, component sharing, and scheduling into a unified framework, effectively improving system throughput while minimizing overall deployment costs, thereby enhancing user experience.
Xue-Jian Chi, Yi-Fei Zou, Cong-Wei Zhang et al.· IEEE Transactions on Mobile...· 1 citation
Morph is a serving system that co-designs the model architecture and runtime for request-adaptive hybrid LLM inference and uses prefill-oriented gating to select a request-specific layout that remains fixed throughout decoding.
Fa-Hao Chen, Xu-Zhe Yan, Qi Zhang et al.· 0 citations
This study delves into large PFMs adaptation in the resource-constrained federated learning environment, and proposes an innovative framework, namely ADRAP, which alternates between large model distillation and resource-adaptive pruning, with guaranteed convergence.
Xiao Zhang, Yang-Yang Wang, Xing-Yu Sun et al.· IEEE Transactions on Pattern...· 0 citations
NLCC, a node-level congestion control framework that configures a set of static CCA parameters shared by flows on a node and regulates the aggregate sending rate of a node, is proposed and applied to BBR and evaluated in CDN production environments.
Peng-Qiang Bi, Meng-Bai Xiao, Guang-Hui Zhang et al.· Proceedings of the ACM on Ne...· 0 citations
Modern large language model (LLM) inference suffers from severe Time-To-First-Token (TTFT) bottlenecks. Existing prefix KV caching mechanisms are inherently stateless, forcing a trade-off between cross-chunk attention accuracy and online recomputation overhead. To address this issue, we propose Pegasus, a novel statefu...
Fahao Chen, Peng Li, Dongxiao Yu et al.· Fall Joint Computer Conferen...· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.