The key is that migrating CPU-tolerant models to CPU during bursts can reduce overall GPU interference more than the local slowdown incurred, thus protecting performance-critical GPU models even with slower CPU execution.
Shi-Jie Peng, Yanying Lin, Cheng-Zhi Lu et al.· Proceedings of the Internati...· 0 citations
C cumulative ablations show that adding the three mechanisms successively increases geometric-mean throughput, while budget adjustment improves accepted-token retention, and DScale is presented, preserving drafter architecture, weights, and full draft length.
Rong-Jian Chen, Min-Xian Xu, Zheng-Xin Fang et al.· 0 citations
Inspired by the brownout paradigm in service computing, BrownoutMoE reorganizes experts into groups to improve utilization and system efficiency while maintaining service quality and introduces a grouping-consistent distillation process to produce deployable models that are compatible with standard inference pipelines.
Yi Ding, Minxian Xu, Zheng-Xin Fang et al.· arXiv.org· 0 citations
Cremes is proposed, an adaptive and cost-efficient scaling framework that ensures microservice recovery within the spot instance grace period and maintains SLO violation rates under preemptible environments below 6.7%.
Liao Chen, Chenyu Lin, Junlin Chen et al.· IEEE International Symposium...· 0 citations
This paper explores a movable-antenna (MA) assisted energy minimization scheme in an uncrewed aerial vehicle (UAV)-enabled MEC system, where a UAV equipped with an MA array serves as an edge server to process tasks offloaded from terrestrial consumer electronics devices.
Jiang Chen, Chunjie Wang, Xuhui Zhang et al.· IEEE transactions on consume...· 0 citations
CoScale is proposed, a layer-wise dynamic scaling mechanism that selectively expands the parallelism of hot layers onto idle resources reclaimed from underutilized devices, enabling elastic data parallelism without altering model architectures or adding hardware overhead.
Jingfeng Wu, Yiyuan He, Minxian Xu et al.· 0 citations
Large language model (LLM) deployment at the network edge faces a fundamental paradox: applications require full-scale models for sophisticated reasoning, yet edge devices impose severe resource constraints across computation, memory, and network. Existing approaches fail to effectively orchestrate resources across the...
Yanying Lin, Baicheng Chen, Xinyu Zhang et al.· International Symposium on C...· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.
