Large language models (LLMs) increasingly run on edge devices alongside tasks that continuously process sensor data. In many applications, these foreground tasks prepare observations as inputs to the LLM, whose outputs guide subsequent processing or actions. The observations must be processed by foreground deadlines, a...
Teng Mei, Cheng-Xuan Pei, Wei-Jun Wang et al.· 0 citations
CALM-MAS is proposed, a congestion-aware serving framework for LLM applications that treats LLM test-time computation as an elastic resource, dynamically adjusting the compute profile of admitted tasks to tame congestion.
Mouheb Ben Nasr, Muhammad Bilal, Alessandro Cornacchia et al.· Proceedings of the 17th ACM...· 0 citations
Preliminary but compelling evidence that MC-aware scheduling is a practical operating system/runtime direction for deployable embodied AI on unified memory edge devices is provided.
Teng Mei, Cheng-Xuan Pei, Marco Canini et al.· Proceedings of the 17th ACM...· 2 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.