2025
DynaPipe: Dynamic Layer Redistribution for Efficient Serving of LLMs with Pipeline Parallelism
DynaPipe is a dynamic layer redistribution scheme that adaptively balances computation by predicting execution latency in real time that reduces average end-to-end request latency by 8% to 41% across diverse workloads, outperforming state-of-the-art pipeline parallelism systems.
Hongxin Xu, Tianyu Guo, Xianwei Zhang
· Neural Information Processin... · 2 citations