POSTER: Prediction-Enhanced Expert Prefetching and Eviction for MoE Offloading via PRED-MoE
PRED-MoE uses a lightweight predictor to assess which experts are most likely to be required for prefetching and ranks experts by their near-future activation likelihood for eviction, and indicates that it reduces the TPOT of vLLM and HuggingFace by up to 3.02× and 28.62× respectively.