Is Your NPU Ready for LLMs? Dissecting the Hidden Efficiency Bottlenecks in Mobile LLM Inference
This study presents the first comprehensive, cross-layer measurement study of mobile LLM inference, uniquely spanning five mainstream frameworks and three hardware backends, and identifies a distinct phase split where NPUs excel at compute-bound prefilling, while CPUs outperform all other backends in memory-bound decod...