This analysis supports a correctness-prediction bottleneck hypothesis: current routers primarily learn global-average model performance trends rather than fine-grained, query-specific routing signals, and collectively fail on queries that require instance-specific routing decisions.
Yi-Fan Lu, Qi-Yue Zhang, Shen-Run Zhang et al.· 0 citations
RegEXROUTE is introduced, a pipeline that uses sparse autoencoders (SAEs) to discover interpretable regular-expression (regex) features and establishes explicit, interpretable text features as a practical basis for designing and understanding LLM routers.
GPUs are expensive resources but often underutilized. This inefficiency is exacerbated by the increasing diversity of AI workloads, where small or specialized models often fail to saturate high-capacity devices, necessitating GPU sharing. Existing sharing approaches force a hard tradeoff. They either partition hardware...
Yi-Cheng Liu, Yi-Fan Qiao, Tian Xia et al.· Proceedings of the 32nd ACM...· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.