GPUs are expensive, yet inference-serving GPU clusters remain heavily underutilized. To improve utilization, state-of-the-art systems adopt GPU multiplexing. However, optimizing solely for utilization can counterintuitively increase energy consumption. Designing policies that treat power and energy as first-order metri...
Prasoon Sinha, Dimitrios Liakopoulos, Nathan Lemma et al.· Proceedings of the ACM SIGOP...· 0 citations
It is found that fewer tokens need not mean faster or cheaper execution: on Terminal-Bench with Qwen, policies using roughly one-third as many tokens can take 20-80% longer than the uncompressed agent.
Ritul Satish, Prasoon Sinha, Akiho Kawada et al.· 0 citations
MaverIQ is an intent-based LLM inference serving system that automatically maps user intents to deployment configurations while minimizing operational cost for the provider and reduces profiling overheads by 7-15× compared to state-of-the-art baselines.
Dimitrios Liakopoulos, Prasoon Sinha, Tianrui Hu et al.· ACM SIGOPS Operating Systems...· 0 citations
This work presents Oneiros, a dynamic remapping engine for multi-tenant LLM serving that dynamically repurposes GPU memory allocated for model parameters as KV cache capacity, enabling nonblocking, unidirectional parameter transfer.
