Jul 2026
HeteroMosaic: Exposing and Exploiting Heterogeneous Execution Opportunities for Energy-Efficient Edge LLM Inference
HeteroMosaic first uses a heterogeneous roofline model to identify when combining iGPU and NPU execution is beneficial and decomposes inference into dependency-preserving micro-batches that expose cross-accelerator overlap and applies trace-guided co-optimization of scheduling and device allocation under practical effects such as memory contention, DVFS, device variation, and NPU runtime overheads.
Gregory Jun, Wesley Pang, E. Richter et al.
· arXiv.org · 0 citations