W. Jung

TIDE: Temporal Incremental Draft Engine for Self-Improving LLM Inference

TIDE (Temporal Incremental Draft Engine), a serving-engine-native framework that integrates online draft adaptation directly into high-performance LLM inference systems, and reuses target model's intermediate hidden states generated during inference as training signals for draft adaptation, thereby avoiding additional...