Zhao-Geng Li

Adapting Tree-Structured Speculative Decoding to DeepSeek-V4 for Efficient Inference

Repeated execution of the target model during autoregressive decoding is a major source of LLM inference latency. Unlike linear speculation, which follows a single candidate chain, tree-structured speculation retains multiple branches from shared prefixes; under the same budget, this broader coverage can improve accept...