CALM-MAS is proposed, a congestion-aware serving framework for LLM applications that treats LLM test-time computation as an elastic resource, dynamically adjusting the compute profile of admitted tasks to tame congestion.
Mouheb Ben Nasr, Muhammad Bilal, Alessandro Cornacchia et al.· Proceedings of the 17th ACM...· 0 citations
The proposed Multi-Path Multi-Level Feedback Queueing (MP-MLFQ) leverages the spatial diversity and regularity of DCNs to realize a scheduler with numerous logical priority levels while occupying as low as 2 physical priority queues within network switches.
Alessandro Cornacchia, Andrea Bianco, Paolo Giaccone et al.· 0 citations
The proposed Multi-Path Multi-Level Feedback Queueing (MP-MLFQ) leverages the spatial diversity and regularity of DCNs to realize a scheduler with numerous logical priority levels while occupying as low as 2 physical priority queues within network switches.
Alessandro Cornacchia, Andrea Bianco, Paolo Giaccone et al.· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.