Large language models (LLMs) increasingly run on edge devices alongside tasks that continuously process sensor data. In many applications, these foreground tasks prepare observations as inputs to the LLM, whose outputs guide subsequent processing or actions. The observations must be processed by foreground deadlines, a...
Teng Mei, Cheng-Xuan Pei, Wei-Jun Wang et al.· 0 citations
Decision models offer fast choices over sup- plied alternatives, while general-purpose large language models interpret open-ended tasks through flexible generation and reasoning at the cost of sequential decoding. Many work- loads expect a generative backend to determine what can be done and therefore provide no well-d...
Cheng-Xuan Pei, Zi-Yao Wang, Teng Mei et al.· 0 citations
This study describes GPU-fabric contention in LLM workloads and presents GPUWeaver, a context-aware communication scheduler that monitors application progress and fabric state, identifies contention, and dynamically regulates communication injection according to application performance goals.
Dan-Yang Chen, Yu-Feng Gu, Yi-Bo Huang et al.· Proceedings of the 17th ACM...· 2 citations
Preliminary but compelling evidence that MC-aware scheduling is a practical operating system/runtime direction for deployable embodied AI on unified memory edge devices is provided.
Teng Mei, Cheng-Xuan Pei, Marco Canini et al.· Proceedings of the 17th ACM...· 2 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.