Preprint
Sep 2026
Cross-Model Autoscaling for Shared LLM Serving
TRE introduces Token Service Share (TSS), a calibrated, demand-normalized signal that estimates effective token service per active or queued request and yields a comparable health score across heterogeneous models and SLO classes.
Xin Zhang, Xian-Yan Xie, Zhen He et al.
· 0 citations