CFMoE: Cache-Friendly Mixture of Experts Model Serving System on Edge Devices
Deploying large language models (LLMs) on edge devices has garnered significant attention due to their inherent privacy and security benefits. However, the enormous parameter scale of modern LLMs poses fundamental challenges for resource-constrained edge devices. Mixture-of-Experts (MoE) models emerge as a promising so...