Jul 2026
Lossless but Not Free: An Empirical Anatomy of Speculative Decoding on Consumer Hardware
A from-scratch, device-agnostic (CUDA/MPS/CPU) implementation and an empirical study across five draft/target backend configurations on a consumer Apple-silicon laptop, finding the best configuration reaches a measured wall-clock speedup.
Param Chordiya
· arXiv.org · 0 citations