Jul 2026
A Sparse Glimpse of the Whole: Train-Free Self-Speculative Decoding
A unified efficiency analysis is presented showing that extending the speculation horizon can reduce rather than improve speedup when the marginal acceptance probability falls below the relative drafting cost, and SparseSpec-L, a training-free self-speculative decoding framework for long-context inference is introduced...
Yue Liu, Yuan Zeng, Min Lyu et al.
· arXiv.org · 0 citations