Janus is a Service-Engine co-designed multi-model serving system built around dual-timescale scheduling that maintains 0.97-1.0 SLO attainment, versus 0.80-0.92 for the strongest baseline, while using 13,440 device-hours/day, 27% less than static Serverless-LLM.
Tian-Bao Zhou, Yi Wang, Yu Zhou et al.· Proceedings of the ACM SIGOP...· 1 citation
FlowTurbo is presented, a self-adaptive, system-level offloading approach that implements hit-driven MegaFlow hardware offloading by jointly optimizing software rule scheduling and hardware rule lookup by jointly optimizing software rule scheduling and hardware rule lookup.
Zhongxian Liang, Sheng Lan, Ying Li et al.· Conference on Applications,...· 0 citations
Dragonfly-Ultra is presented, a scalable, low-cost network architecture for high-performance AI clusters that can scale to over 260k GPUs with only 82% cost and 81% power consumption of a 3-layer Clos architecture and incorporates three key mechanisms to further improve network performance and optimize collective commu...
Rui Zhuang, Hui Yuan, Junye Zhang et al.· Conference on Applications,...· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.
