Cache-Aware Joint Router Adaptation for Memory-Efficient MoE Inference
This work formulate expert-cache management as a model-side algorithmic problem and proposes cache-aware post-training that jointly adapts the MoE backbone and lightweight auxiliary routers while preserving the native inference-time Top-K rule.
Zhen-He Wu, Ya-Ping Jin, Qing-Hua Xing et al.
· 4 citations