Janus is a Service-Engine co-designed multi-model serving system built around dual-timescale scheduling that maintains 0.97-1.0 SLO attainment, versus 0.80-0.92 for the strongest baseline, while using 13,440 device-hours/day, 27% less than static Serverless-LLM.
Tian-Bao Zhou, Yi Wang, Yu Zhou et al.· Proceedings of the ACM SIGOP...· 2 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.