High-quality large language model inference is increasingly used on mobile edge, from satellites to smart homes, where limited connectivity or privacy requirements motivate local model inference. In many such settings, token generation is inherently compute-limited and added latency may be acceptable, but output qualit...
Jun You, Kun Wang, Jie-Song Chen et al.· Proceedings of the 32nd Annu...· 0 citations
Edge applications increasingly combine CNN-based perception with edge-deployable Transformer-based language models, but deploying such heterogeneous workloads on resource-constrained devices is challenging because inference latency varies with both input difficulty and runtime hardware state. Early-exit inference reduc...
Yu-Hang Guan, Huan-Le Zhang, Xiao-Min Ouyang et al.· 0 citations
To bridge the gap between reasoning depth and real-time constraints, Astra incorporates an asynchronous synergetic protocol featuring strategic locking for stability and proactive feedback for adaptive re-planning, offering superior efficiency, robustness, and generalization across diverse urban scenarios.
Fansheng Sun, Ji-Yu Wang, Zhidan Liu· Proceedings of the 32nd ACM...· 0 citations
To bridge the gap between reasoning depth and real-time constraints, Astra incorporates an asynchronous synergetic protocol featuring strategic locking for stability and proactive feedback for adaptive re-planning, offering superior efficiency, robustness, and generalization across diverse urban scenarios.
Fansheng Sun, Jiyu Wang, Zhidan Liu· Proceedings of the 32nd ACM...· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.