Aug 2026· 1 citation· ⚡ 1 influential· 30 references
Computer Science
TL;DR
Results show that ASR self-speculation depends on token prediction, audio-position tracking, and draft cost, and AnchorDraft, which teaches the draft to track the audio position during training without changing the inference graph, improves end-to-end speed at both tested target scales.
Abstract
Speculative decoding speeds up generation by letting a cheap draft propose several tokens that a target model checks in one pass. In the single-model form, the draft is a lightweight module attached to the target rather than a separate model. Applying this design to Automatic Speech Recognition (ASR) introduces an extra problem. The draft can read the whole audio at every step, yet its proposals get worse as it runs on its own. Access is not localization. The accepted text keeps the transcript position explicit, but the draft must also track the changing audio position. In the primary matched comparison, per-step audio access changes the first proposal modestly but roughly doubles later-proposal acceptance. Fixed-width windows show that the audio position explains part of this gap. A correctly placed window recovers continuation, while an equally narrow window at the wrong position reduces it. Late-draft median error reaches 21 frames in the hardest reported condition, while target attention during verification stays within a 2-frame median. We test two ways to reduce this drift. The first reads the audio position from verification attention and uses it to guide the next draft round. It saves time only when the extra accepted tokens offset the readout cost. The second is AnchorDraft, which teaches the draft to track the audio position during training without changing the inference graph. The trained draft improves end-to-end speed at both tested target scales. These results show that ASR self-speculation depends on token prediction, audio-position tracking, and draft cost.
GLANCE is presented, the first one-pass block drafter that is lossless on an unmodified VLM target, and it breaks the cycle at both ends of a self-defeating cycle.
Jungseob Lee, Seongtae Hong, D. Lee et al.· 0 citations
Verification-Aware Training is introduced, a plug-in framework that simulates verification at every training step and turns the resulting accept and reject patterns into supervision and improves average acceptance length and wall-clock speedup across math, code, and chat benchmarks.
Geonmo Gu, Byeongho Heo, HeeJae Jun et al.· 6 citations· ⚡1
The results show that source-conditioned audio generation can relax the conventional dependence of speculative drafting on the target's evolving output prefix, exposing substantial parallelism for efficient inference.
Yun-Zhe Li, Kyoungjun Park, Hong-Zi Zhu et al.· 0 citations
This work introduces LongSpark, a block-diffusion drafter that achieves state-of-the-art end-to-end efficiency across multiple model scales and realistic serving conditions by extracting fixed-size, multiscale views from the target's verification pass, thereby eliminating the need for a growing persistent state.
Hao-Yuan He, Peng-Fei Liu, Si-Shi Shen et al.· 0 citations
LiLiCorr, a Lightweight Likelihood-based model that Correlates the per-position marginal distributions a drafter already produces, is introduced, a Lightweight Likelihood-based model that Correlates the per-position marginal distributions a drafter already produces.
M. Rusanovsky, Yoav Miron, Roy Uziel et al.· 4 citations
DBloom speeds up generation with an efficient draft model (drafter) that proposes tokens for a target model to verify in one pass, preserving the target's output distribution and recommending the histogram as a preflight check before spending training compute.
Ephrem C. Wu· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.