TAILOR: Token-Aware Partitioning and Routing for Edge–Cloud Transformer Inference
TAILOR is presented, a token-aware inference framework that jointly optimizes offline partitioning and online routing for autoregressive transformer serving that reduces end-to-end latency and lowers OOM-induced fallback compared with static partitioning baselines, demonstrating robust inference under long-tailed token...