Or OrionInfer, an adaptive LLM serving system that aligns inference strategies with real-time demand and introduces three key techniques: runtime switching between data parallelism and tensor parallelism with negligible overhead, an efficient inference pipeline that preserves batching efficiency during parallelism tran...
Jingqi Feng, Guang Yang, Yukai Huang et al.· Proceedings of the 32nd ACM...· 0 citations
Or OrionInfer, an adaptive LLM serving system that aligns inference strategies with real-time demand and introduces three key techniques: runtime switching between data parallelism and tensor parallelism with negligible overhead, an efficient inference pipeline that preserves batching efficiency during parallelism tran...
Jingqi Feng, Guang Yang, Yukai Huang et al.· Proceedings of the 32nd ACM...· 0 citations
This work advocates for Joint Online-Offline Fine-Tuning as a superior paradigm that breaks the convention of restricting offline data to SFT and online data to RFT, and provides the first comprehensive survey focusing specifically on the synchronization of data provenance.
Tai-Hang Zhen, Guang Yang, Chen-Zhang Li et al.· Proceedings of the Thirty-Fi...· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.
