Decode-Latency Feedback Prefill: A Model-Free Controller and Its Generalization Limits
Decode-Latency Feedback Prefill is introduced, a model-free controller that changes only prefill work that overlaps active decodes that defines the boundary of the contribution and motivates a completion-timed controller for concurrent CPU and on-device inference.