WAQ-LLM is proposed, a performance optimization framework to find the optimal deployment configuration for multi-instance LLM serving that can provide adaptive and hybrid configurations to handle diverse workloads, surpassing static deployments limited to a single instance type.
Jia-Xin Lai, Yi-Zhou Luo, Qiang Wang· Proceedings of the Internati...· 0 citations
WAQ-LLM is proposed, a performance optimization framework to find the optimal deployment configuration for multi-instance LLM serving that can provide adaptive and hybrid configurations to handle diverse workloads, surpassing static deployments limited to a single instance type.
Jia-Xin Lai, Yi-Zhou Luo, Qiang Wang· Proceedings of the Internati...· 0 citations
Parallax is presented, a fine-grained and accurate performance prediction framework tailored for training–inference co-execution and introduces interpretable modeling strategies for the two primary GPU sharing paradigms.
Zesen Hu, Desheng Wang, Sichao Chen et al.· ACM Transactions on Architec...· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.