Balancing and Beyond: Communication-Centric Optimizations in Expert Parallelism
EPIC mitigates imbalance via performance-aware expert migration and runtime expert activation, and then improves communication with topology-adaptive transport kernels and fine-grained computation-communication overlap.
Jia-Min Cao, Qingxu Li, Yaozhong Liu et al.
· Conference on Applications,... · 2 citations
· ⚡1