Deploying large language models (LLMs) on edge devices has garnered significant attention due to their inherent privacy and security benefits. However, the enormous parameter scale of modern LLMs poses fundamental challenges for resource-constrained edge devices. Mixture-of-Experts (MoE) models emerge as a promising so...
Hao-Yu Zhang, Hong-Yi Wang, Jian-Wei Zheng et al.· Proceedings of the 32nd Annu...· 0 citations
Cloud execution of mobile apps—running unmodified apps inside full mobile OSes on commodity servers—is gaining adoption for uses such as GUI agent training and large-scale app testing. These workloads are both temporally dynamic (from second-scale agent interactions to hour-long tests) and highly bursty (up to hundreds...
Jia-Xing Qiu, Zi-Jie Zhou, Zhen-Hua Li et al.· Proceedings of the ACM SIGOP...· 0 citations
LMTracer is presented, a fine-grained and real-time performance profiling framework for production LLM services that embed profiling logic into the execution through graph-embedded probing and streaming buffered profiling data to CPUs on demand to keep the execution of user kernels uninterrupted.
Wei Liu, Yong-Chao He, Bo-Han Zhao et al.· Proceedings of the ACM SIGOP...· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.