This work presents Whisper-Flash, a two-layer drafter built on a property of speech recognition: the words still to be written have already been spoken, which reads Whisper's encoded audio and accepted decoder states and proposes eight tokens in a single forward pass.
Hua-Peng Zhou, Hua-Yu Wang, Junkai Wu et al.· 0 citations
TreeSpark is introduced, which reads a parent-conditioned distribution from the drafter's existing Markov head at negligible cost, calibrates it into an edge-acceptance estimate, and lets path survival govern everything else: best-first expansion, per-round stopping, and a load-adaptive serving policy.
Results show that ASR self-speculation depends on token prediction, audio-position tracking, and draft cost, and AnchorDraft, which teaches the draft to track the audio position during training without changing the inference graph, improves end-to-end speed at both tested target scales.