A Playground for Benchmarking Agentic AI in Network Management

Abstract

Agentic AI is emerging as a promising paradigm for network management, enabling high-level intent processing, automated decision making, and closed-loop control. However, the current landscape is fragmented: proposed solutions are often evaluated in ad hoc settings, with limited reproducibility and no common basis for systematic comparison. This lack of benchmarking methodology makes it difficult to assess the actual benefits, limitations, and operational trade-offs of different agentic approaches. This paper presents a playground for benchmarking agentic AI in network management. Rather than proposing a single best-performing agent, the goal is to provide a controlled and extensible environment in which heterogeneous agentic solutions can be deployed, observed, and compared under common network management tasks. The playground combines a programmable multi-node network topology, a transaction-oriented control workflow, structured agent-to-network interfaces, explicit network state representation, and built-in validation and rollback mechanisms. Its design enforces a clear separation between high-level agent reasoning and deterministic execution, thus enabling safer and more auditable experimentation. The proposed framework is instantiated over a network management scenario based on Segment Routing over IPv6 (SRv6), where agentic solutions interact with the infrastructure through declarative messages instead of arbitrary low-level commands. This design supports benchmarking along multiple dimensions, including task success, convergence behavior, robustness to failures, recovery capability, safety of issued actions, and auditability of the control process. By providing a reproducible and observable experimentation environment, the proposed playground lays the foundation for a systematic evaluation methodology for agentic AI in network management.