As LLMs scale, mixture-of-experts (MoE) architectures increase model capacity while keeping per-token computation manageable through sparse expert activation. This sparsity also enables memory-efficient deployment by retaining most expert weights in CPU memory and transferring or executing only the routed experts on de...
This work analyzes 118,000 skills and proposes SkillVM, a compilation and runtime system designed for portable and efficient skill execution that achieves up to 3.2× speedup with enhanced parallelism, and 19-50× latency reduction through code solidification.
Le Chen, Er-Hu Feng, Yu-Bin Xia et al.· Proceedings of the ACM SIGOP...· 0 citations
WaferAI-SIM enables simulator-driven co-design of LLM serving strategies and multi-core accelerator architectures, targeting the early design stage in which emerging platforms are not yet broadly available for empirical serving studies, and provides guidance for co-designing hardware architectures and serving strategie...
Tian-Hao Zhu, Da-Hu Feng, Er-Hu Feng et al.· IEEE Transactions on Compute...· 2 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.
