Aug 2026· Asia-Pacific Workshop on Networking· 0 citations· 36 references
Computer Science
TL;DR
An architecture that decouples complex policy enforcement from high-speed packet forwarding to support VPC semantics on back-end NICs and enable front-end/back-end integration is proposed, suggesting that commodity hardware can support both high-throughput AI training and flexible VPC features.
Abstract
While commodity NIC-based back-end AI networks offer ultra-high intra-cluster bandwidth for distributed training, their limited programmability and on-chip resources hinder the implementation of advanced VPC features such as fine-grained isolation and stateful security policies. Furthermore, access to resources within the VPC needs to be routed through the front-end DPU, which is shared by the scale-up domain. The mismatch between the front-end DPU’s bandwidth and the back-end requirements causes GPU underutilization when intensive VPC communication is required for content recommendation, AIGC, and federated learning workloads. We propose an architecture that decouples complex policy enforcement from high-speed packet forwarding to support VPC semantics on back-end NICs and enable front-end/back-end integration. Evaluations show near-full GPU utilization in our analytical model and 71 μ s P999 extra latency of the first packet, suggesting that commodity hardware can support both high-throughput AI training and flexible VPC features.
The experience in designing, deploying, and operating Pegasus, a data center network tailored for the AI cloud, along with operational lessons learned from its deployment are shared.
Xianneng Zou, Yadong Liu, Yiran Zhang et al.· Conference on Applications,...· 0 citations
A standards-based, multi-tenant cloud inference framework that integrates OpenStack orchestration with Single Root I/O Virtualization (SR-IOV)-enabled graphics processing unit (GPU) partitioning to achieve predictable and isolated real-time inference execution.
Rui Ma, Bingfeng Shi, Xin Ma et al.· Journal of ICT Standardizati...· 0 citations
The rapid proliferation of Artificial Intelligence (AI) workloads, including Large Language Models (LLMs) and Generative AI (GAI), is fundamentally transforming Multi-access Edge Computing (MEC) infrastructures. AI-driven services deployed at the edge exhibit highly synchronized, bursty and bandwidth-intensive traffic patterns, contrasting with traditional MEC applications that generate moderate, stable flows. This evolution stresses intra-MEC network fabrics, leading to potential congestion, unfair queuing and underutilized computational resources when conventional architectures designed for elastic workloads are applied. In response, we propose an AI-aware intra-MEC data center network (DCN) architecture coupled with Wavelength Division Multiplexing (WDM)-based Medium Access Control (MAC) protocols. The proposed edge DCN enables fine-grained traffic prioritization, ensuring latencysensitive AI workloads maintain sub-us latencies while efficiently accommodating medium and low-priority applications. Through simulation-driven evaluation under realistic AI escalation scenarios, we demonstrate over 90% bandwidth utilization with near-zero packet loss, deterministic latency guarantees across heterogeneous workloads, and robust performance even under extreme AI traffic bursts. These results highlight the potential of AI-aware, intra-MEC network architectures to meet the demands of AI-centric edge environments. Ultimately, this work contributes to the realization of next-generation networks and the MEC 2.0 vision, where intelligent, high-performance edge infrastructures seamlessly support diverse, latency-sensitive workloads at scale.
G. Drainakis, P. Baziana, A. Bogris· International Mediterranean...· 0 citations
STREAM (Smart Tiered Routing Engine for AI Models) addresses the gap between local, HPC, and cloud inference with a three-tier routing architecture combining local, HPC, and cloud inference with a local LLM-based complexity judge.
A. Nassar, Steve Mohr, L. Apanasevich et al.· Practice and Experience in A...· 0 citations
Compute and memory resources in cloud environments are strictly managed and isolated by the control plane; in contrast, network resources lack equivalent management and isolation mechanisms. This best-effort treatment of networking leads to significant challenges for modern AI workloads, which have diverse and bandwidth-intensive communication patterns. Without fine-grained network resource control, these workloads suffer from interference, unpredictable throughput, and suboptimal cluster utilization. To address these issues, this paper demonstrates how network bandwidth can be elevated to a first-class, schedulable, and enforceable resource within Kubernetes, the de facto standard for cloud-native orchestration. We introduce a new scheduling capability that models network interfaces as allocatable resources and regulates bandwidth sharing through the Dynamic Resource Allocation (DRA) framework, with enforcement implemented using the Hierarchical Token Bucket (HTB) mechanism. We evaluate the system using multitenant AI workloads derived from real-world communication characteristics with a simulation-based approach and validate the proposed enforcement strategy in a real cluster. Results show that the proposed two-level bandwidth allocation improves tenant performance predictability and satisfaction while maintaining packed cluster utilization.
Gleam is a novel and network-efficient framework for task-generic GPU sharing across local-area CUDA devices, with three key contributions that reduce bandwidth overhead in CUDA API remoting through automatic model weight caching, and mitigate accumulated latency from frequent API calls by asynchronous execution.
