Meixuan Wang

LLM Serving Optimization with Variable Prefill and Decode Lengths

Sorted-F, which repeatedly selects feasible batches using an F-metric that balances batch cardinality against downstream decode cost, achieves a constant-factor approximation guarantee in the unit-time, uninterrupted-decoding model with known output lengths.