Hogun Seong

Target Regeneration Shrinks the Gains of Draft-Training Objectives in Speculative Decoding

Speculative decoding accelerates large language models by verifying drafted tokens in parallel. New draft-training objectives are usually credited with the acceptance they add over cross-entropy on externally written responses, yet many pipelines let the target rewrite them. We ask how much of that gain survives. With...