Yuanbo Tang

Multi-level context Modeling for consistent expert selection in Mixture-of-Experts

This work revisit expert selection from a representation perspective and identifies context incompleteness as a key bottleneck limiting effective expert specialization, proposing Multi-level Context Fusion MOE (MCF-MOE), a framework that constructs context-aware representations by integrating complementary signals from cross-layer semantic aggregation and local token-level interactions.