Skip to content
Log in
Sponsored AI Networking Cookbook: Practical recipes for AI-assisted network automation and development AI Networking Cookbook: Practical recipes for AI-assisted network automation and development
Book Open access

Scaling LLM Agent Tool Access at Cloud Scale

Ming-Xing Li Enge Song Yueshang Zuo Xiaodong Liu Rong Wen Jian-Bao He Tie Jing Zhou Shao Qiang Fu Xiaobo Xue Xiong Xiao Luya Zhong Shao-Kai Zhang Jian-Gu Zhao Jian-Yuan Lu Shize Zhang Xiaoqing Sun Chang-Gang Zheng Zi-Hao Fan Hao-Nan Li Tian Pan Xiao-Min Wu Yang Song Xing Li Biao Lyu Meng Li Hai-Peng Dai Guihai Chen Shunmin Zhu
Aug 2026 · Asia-Pacific Workshop on Networking · pp. 304-307 · 0 citations · 6 references
Computer Science

TL;DR

The gateway breaks the direct-connect data plane and consolidates legacy API integration, protocol bridging, access control, and session-aware routing, while scaling out elastically at low per-call overhead.

Abstract

LLM agents increasingly rely on tool calling, and the Model Context Protocol (MCP) standardizes it between agents and tool providers, reducing integration cost and driving rapid growth in tool scale. Yet a standardized interface does not make tool access work at production scale: legacy services are not MCP-callable, fast protocol evolution creates compatibility cost, large tool sets exhaust the context window, and stateful sessions complicate load balancing. We solve these with a shared control point, a centralized MCP Gateway System that makes MCP operational at cloud scale. The gateway breaks the direct-connect data plane and consolidates legacy API integration, protocol bridging, access control, and session-aware routing, while scaling out elastically at low per-call overhead. It scales agent tool access to thousands of cloud operations.

DOI Semantic Scholar
Read PDF

Similar papers

Preprint Aug 2026

Hybrid Semantic Tool Discovery for Enterprise MCP Gateway: Architecture and Implementation

This work presents SCOUT (Selective Context Optimization for Universal Tooling for Universal Tooling), which reframes tool exposure as a context-selection problem, injecting only tools relevant to the current step, and reduces MCP tool-token consumption by 99%, cutting per-query inference cost at enterprise scale.

Olympia Saha, Amy Wang, Srinivasan Manoharan · 0 citations
Open access Sep 2026

FinDS-Agent: A Cloud–Edge Collaborative Data Science Agent for Financial Analytics

FinDS-Agent is presented, a cloud–edge framework that keeps raw records and program execution at the trusted edge while providing a policy-screened, sanitized context to support cloud planning and support selective cloud planning while delimiting statistical, privacy, and transfer claims.

Xiao-Zheng Du, Rui-Jun Deng, Cheng Wang et al. · 0 citations
#artificial intelligence Preprint Sep 2026

Carbon-Aware Routing for Function Calling in Edge-Cloud LLM Systems

A carbon-aware routing framework that distributes function-calling queries across a three-tier edge-cloud architecture, combining edge and cloud LLMs on heterogeneous hardware and matches cloud-level accuracy while reducing operational carbon emissions by $4\times on average.

Aikaterini Maria Panteleaki, Varatheepan Paramanayakam, S. Tragoudas et al. · 1 citation
Aug 2026

OMBench: Taming Data Management Requirements in Cloud-Native Applications

This work presents OMBench through a decision-and-evaluation toolkit with rich visualization features so that users can navigate a set of candidate platforms to meet data management requirements and make informed decisions about their guarantees and resource implications.

R. Laigner, Xikun Jiang, Boris Düdder et al. · 0 citations

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.