Preprint
Aug 2026
Depth-adaptive Inference of Looped Language Models via Continuous Depth Batching
Continuous depth batching (CDB) is introduced, which schedules at the granularity of individual loop iterations, and handles boundary stages and loop steps in separate priority queues, makes exit decisions one step ahead, and overlaps all scheduling work with GPU computation.
Kristian Schwethelm, D. Rueckert, G. Kaissis
· 0 citations