TeDiServe
TeDiServe is a cluster-level serving system for diffusion language models that targets high latency-SLO attainment through deadline-aware scheduling, confidence-threshold-based load control, and cluster reconfiguration.
Tzu-Tao (Tommy) Chang, Benjamin Yuanyang Hong, Kiet Pham et al.
· Zenodo (CERN European Organi... · 0 citations