Book
Open access
Aug 2026
POSTER: Prediction-Enhanced Expert Prefetching and Eviction for MoE Offloading via PRED-MoE
PRED-MoE uses a lightweight predictor to assess which experts are most likely to be required for prefetching and ranks experts by their near-future activation likelihood for eviction, and indicates that it reduces the TPOT of vLLM and HuggingFace by up to 3.02× and 28.62× respectively.
Wenchen Han, S. Vargaftik, Michael Mitzenmacher et al.
· Proceedings of the ACM SIGCO... · 0 citations