Generative AI workflows execute user tasks by composing sequences of LLM calls, where each call performs a sub-task (e.g., summarization or translation), forming a workflow. Executing such workflows requires decisions over workflow structure, model assignments, and deployment configurations, which jointly determine lat...
Dimitrios Liakopoulos, Prasoon Sinha, N. Yadwadkar· 0 citations
GPUs are expensive, yet inference-serving GPU clusters remain heavily underutilized. To improve utilization, state-of-the-art systems adopt GPU multiplexing. However, optimizing solely for utilization can counterintuitively increase energy consumption. Designing policies that treat power and energy as first-order metri...
Prasoon Sinha, Dimitrios Liakopoulos, Nathan Lemma et al.· Proceedings of the ACM SIGOP...· 0 citations
MaverIQ is an intent-based LLM inference serving system that automatically maps user intents to deployment configurations while minimizing operational cost for the provider and reduces profiling overheads by 7-15× compared to state-of-the-art baselines.
Dimitrios Liakopoulos, Prasoon Sinha, Tianrui Hu et al.· ACM SIGOPS Operating Systems...· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.
