TopoCompress: Topology Aware Token Compression Algorithm for Distributed Edge MoE Inference
Simulations demonstrate that TopoCompress effectively reduces cross-server traffic and deployment resource consumption while maintaining controllable inference quality, enabling efficient distributed MoE inference over bandwidth- and resource-constrained edge infrastructures.
Ning Li, Xin-Yu Wang, Xin Yuan et al.
· 0 citations