UniPrefill: Universal Long-Context Prefill Acceleration via Block-wise Dynamic Sparsification
UniPrefill is proposed, a prefill acceleration framework applicable to virtually any model architecture, which directly accelerates the model's computation at the token level and achieves up to 2.1x speedup in Time-To-First-Token (TTFT), with the acceleration becoming increasingly pronounced as the number of concurrent...