WAQ-LLM: Optimizing Multi-Instance LLM Deployment via Workload-Aware Queueing Model
WAQ-LLM is proposed, a performance optimization framework to find the optimal deployment configuration for multi-instance LLM serving that can provide adaptive and hybrid configurations to handle diverse workloads, surpassing static deployments limited to a single instance type.