To Copy or Not to Copy: Controlling Speculative Decoding via Intrinsic Model Signals
SwitchSD is introduced, an adaptive framework that treats copying as a latent control signal of the LLM that allows the system to dynamically switch between neural drafting and context-based copying, effectively turning copying from a noisy heuristic into a principled, model-aware decoding regime.
Roy Eisenstadt, Ido Cohen, Edo Cohen-Karlik et al.
· 1 citation