Aug 2026· American Journal of Technology· Vol 5, pp. 66-83· 0 citations
TL;DR
These findings demonstrate that deployment decisions based on structural failure domains can substantially limit the propagation of failures beyond the intended canary population, and indicate that topology-aware rollout control and per-datacenter monitoring can substantially reduce failure propagation and containment time relative to topology-blind global monitoring.
Abstract
Aim: This study aimed to extend the staged-rollout risk framework proposed by Zhao et al. (2018) to multi-datacenter environments by accounting for failure-domain topology, per-datacenter sequential monitoring, and automated rollback. The resulting framework, termed the Isolation Deployment Protocol (IDP), was designed to address the limitations of using traffic-percentage exposure as a proxy for failure exposure in distributed systems with shared control-plane services, replicated caches, and cross-region service-discovery dependencies.
Methods: The Isolation Deployment Protocol (IDP) was developed by integrating four components: a real-time Failure Domain Graph Model, a Topology-Aware Ramp-Up (TARU) algorithm, per-datacenter Sequential Probability Ratio Test (SPRT) monitoring, and automated topology-aware rollback. IDP was evaluated on a six-node bare-metal Kubernetes testbed running Istio and Consul, with namespace isolation used to represent separate datacenters. Twenty independent trials were conducted against a topology-blind baseline, measuring realized blast radius and Mean Time to Containment (MTTC). The blast-radius model was also validated through formal analytical analysis.
Results: Formal analysis demonstrated that the topology-blind baseline could underestimate blast radius by up to 8.62× in the evaluated failure scenario. Across 20 independent trials, IDP reduced the mean realized blast radius by a factor of 13.0 compared with the topology-blind baseline, restricting error propagation to the targeted 5% canary namespace. IDP also reduced Mean Time to Containment by 94.4%, from 142.4 minutes under global metric aggregation to 7.9 minutes with per-datacenter monitoring. These findings demonstrate that deployment decisions based on structural failure domains can substantially limit the propagation of failures beyond the intended canary population.
Conclusion: These results indicate that topology-aware rollout control and per-datacenter monitoring can substantially reduce failure propagation and containment time relative to topology-blind global monitoring.
Recommendation: The experiments ran within a single bare-metal host, and production multi-region network latency was not modeled; this represents an open limitation of the current evaluation. Future research should validate IDP across geographically distributed datacenters under realistic inter-region latency, packet loss, traffic variability, and production-scale dependency graphs.
Network slicing is a foundational capability of Fifth Generation (5G)-Advanced and emerging Sixth Generation (6G) networks, yet practical support for seamless runtime slice transitions remains limited. Standard cloud-native 5G architectures lack native support for stateful inter/intra-slice session migration, relying instead on high-overhead Non-Access Stratum (NAS) re-registrations, container redeployment etc., which disrupt userplane traffic for up to 245.50 ms. To address this limitation, we present Orchra, an intelligent orchestrator for stateful, low-latency context transfer. By externalizing critical user equipment state-including NAS context, security keys, and Protocol Data Unit (PDU) session information-into a transient staging layer, Orchra preserves session continuity across slice boundaries without requiring full re-registration. Experimental evaluation shows that Orchra reduces this userplane interruption by more than twice in comparison to conventional Third Generation Partnership Project (3GPP)-based approaches while incurring negligible security overhead. These results demonstrate a practical and reproducible approach for enabling seamless, state-preserving slice transitions in cloud-native 5G-Advanced networks.
Anthony Kiggundu, Bin Han, H. Schotten· 0 citations
Google's Global Network (GGN), a major architectural redesign of the WAN that evolves B2 and B4 into a single, modular, and highly available software-defined network, is presented.
Mohammad Al-Fares, R. Alimi, Arda Balkanay et al.· Conference on Applications,...· 0 citations
In-network congestion control algorithms (CCAs) are essential for emerging datacenter applications to achieve high throughput and low tail latency. Deploying these CCAs in production requires preserving the existing, highly optimized network datapath as much as possible, since rebuilding or modifying the datapath demands substantial engineering effort to regain performance. Although eBPF enables flexible kernel programmability, deploying in-network CCAs through it remains challenging due to semantic misalignment across kernel hooks, compatibility issues with existing mechanisms, and instability across kernel versions. We present eCCA, an eBPF-based framework that enables the deployment of in-network CCAs in the standard Linux kernel without intrusive code modifications. To address the above challenges, eCCA introduces unified context management, functionally correct hook orchestration, and deployment-adaptive abstractions, which are derived from our practical deployment experiences. We demonstrate the practicality of eCCA by deploying CAQM across multiple NIC models and Linux kernel versions with minimal requirements. Evaluations show that CAQM on eCCA preserves native CAQM’s control behavior with less than 3% throughput loss, while reducing tail latency by 96% compared to CUBIC at the same throughput.
Xinshu Wang, Bin Yang, Jian Cheng et al.· Asia-Pacific Workshop on Net...· 0 citations
Multi-provider LLM gateways reverse proxies that route, load-balance, and rate-limit requests across foundation-model APIs have become critical production infrastructure. Yet the failure modes specific to this architectural layer remain undocumented, scattered across issue trackers and post-mortems with no unifying framework. We introduce \fa{}, a two-axis taxonomy that classifies failures by their \emph{origin layer} (Network/Transport, Streaming/Protocol, State/Session, Model~Behavior, Governance/Cost) and their \emph{detectability} (Loud vs.\ Silent). We populate this taxonomy with five verified catalog entries sourced from public bug reports and first-hand stress testing, each accompanied by a mechanistic root-cause analysis. Three entries include standalone reproduction scripts. Our principal finding is that the most operationally severe failures are \emph{silent}: they return HTTP~200, pass every standard health check, and corrupt application state in ways that require semantic-level observability to detect. Two such silent failures a concurrency race condition causing history loss and a streaming index collision corrupting tool-call payloads were discovered first-hand during \cb{} evaluation campaigns.
Driven by the requirements of machine learning, cloud storage, and other network-intensive applications, remote direct memory access (RDMA) has been widely adopted in high-speed networks and is gradually being applied to geographically distributed datacenters. However, in cross-datacenter scenarios, long control loop latency and mixed traffic prevent existing RDMA congestion control schemes from perceiving and reacting to congestion in a timely and fair manner; this can lead to severe performance degradation and unfairness. To address these issues, we propose cdcPIM, a proactive congestion control scheme extended from datacenter parallel iterative matching (dcPIM) for cross-datacenter networks, which restructures the end-to-end control loop by introducing switch-coordinated control points, effectively transforming long-haul, RTT-bound feedback into localized control. Specifically, cdcPIM deploys a local control point by moving the token generation from the receiver to the sender side cross-datacenter switch, constraining the congestion control loop for inter-datacenter traffic within a single datacenter. Furthermore, cdcPIM introduces a remote control point to perform admission control for inter-datacenter traffic entering the receiver’s datacenter, thus avoiding intra-datacenter congestion caused by traffic bursts. Simulations demonstrate that when cdcPIM manages inter-datacenter traffic while cooperating with datacenter quantized congestion notification (DCQCN) for intra-datacenter traffic, long-haul congestion is effectively mitigated. Under mixed cross-datacenter workloads, DCQCN + cdcPIM reduces the overall average flow completion time (FCT) slowdown by up to 25.7% and the P99 FCT slowdown of intra-DC flows by up to 65.0% compared with the baseline scheme Themis.
Wenqiang Deng, Junyan Chen, Xuefeng Huang et al.· Journal of King Saud Univers...· 0 citations
Consistent replication of data among distributed processes -- a task involving the well-known consensus problem -- is notoriously expensive and hard to scale, affecting especially datacenter services with stringent performance requirements. To mitigate this problem, we introduce scalable replication in-hardware ( scarHW ): a network card design that improves throughput and latency of consistent replication even when increasing the number of replicas, whereas current systems operate at a small scale or with relaxed consistency guarantees. At the heart of scarHW is our novel POPUC consensus algorithm, implemented in an FPGA smartNIC to take full advantage of the"mostly synchronous"behavior of programmable network devices in the datacenter. Unlike widely-adopted"mostly asynchronous"coordination protocols such as Paxos or leaderless alternatives, POPUC implements a generalized variant of consensus dubbed collaborative consensus which allows for several simultaneous decisions, achieving great scalability without compromising availability. POPUC preserves safety guarantees in the presence of process crash-stop and message send/receive omission failures (capturing incidental asynchrony) and has been formally specified and verified in TLA+. Our FPGA prototype improves throughput and latency of widely-used services Redis and Zookeeper by up to two orders of magnitude compared to the state of the art. scarHW-based services also achieve zero downtime upon failure of a minority of replicas, offering a highly-robust, wire-speed, scalable replication system.
Davide Rovelli, Philipp Berdesinski, R. Otoni et al.· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.
