Unified AI Gateway: A Framework for Joint Model Routing and KV Cache Management
This work defines and analyzes the Unified AI Gateway as a system setting for an edge-deployed AI traffic hub that coordinates model routing, KV cache management, and compute placement across end devices, edge resources, and cloud model services.
Jiaxun Lu, Xiang Zhang, Yun-Feng Shao
· 0 citations