Jul 2026
SiFAR: Synchronization-Free All-Reduce for Low-Latency LLM Inference
Speculative reduction is proposed, which initiates data transfer before the top barrier and ensures correctness via lightweight validation during low-latency inference, which reduces synchronization overhead during low-latency inference.
Hritvik Taneja, A. Saxena, Abhishek Revinipati et al.
· arXiv.org · 1 citation