Jia-Yue Wang

OLED-MoE: Accelerating MoE-Based dLLM Inference via Inter-Iteration Locality-Aware Expert Offloading

OLED-MoE is proposed, an expert offloading system that uses confidence-guided inter-iteration prediction to retain high-value experts in GPU memory without introducing extra prefetch traffic and compensates unavoidable cache misses through CPU-GPU cooperative execution.