Anytest: Localizing the Root Cause of Hardware Transport Performance Anomalies

Remote-Timer-as-a-Service: Efficient Microarchitectural Leakage in the Cloud with Remote Timers

Edge computing solutions have become a crucial part of the industry, delivering fast, flexible and scalable applications close to the end users, with typical use cases including dynamic content creation, image resizing and chatbots. Cloudflare Workers is one such framework, which handles millions of HTTP requests per second worldwide. To reduce start-up latency, Cloudflare Workers removes process-isolation boundaries between multiple tenants and leverages language-level isolation. This architecture poses the risk of Spectre attacks. To mitigate these, Cloudflare Workers previously introduced several countermeasures such as restricted timer measurements, no shared memory, no multithreading and Dynamic Process Isolation (DyPrIs), detecting potential attacks and process-isolating potentially malicious scripts. We demonstrate that the production implementation of DyPrIs was insufficient. We adopt microarchitectural amplification techniques and discover various possibilities to measure time in the production environment of Cloudflare Workers. Given these techniques, we show that freezing and coarsening timers in the Cloudflare Workers security model is insufficient. Leveraging both timing amplification and remote timers, we demonstrate a remote Spectre attack that leaks a JWT token from a co-located victim worker in the Cloudflare Workers production environment. We outperform the existing attack by orders of magnitude, going from 2 bit/min to up to 12 bit/s at an accuracy of 99.16%, posing an immediate risk to customer data. Following our end-to-end attack, Cloudflare Workers mitigated it in a coordinated effort by integrating the V8 Sandbox limiting transient access to 64-bit pointers, improving the detection capabilities of DyPrIs, and deploying hardware-assisted MPK-based in-process isolation to confine each tenant heap under a dedicated memory-protection key.

FailureAtlas: A Taxonomy of Failure Modes in Multi-Provider LLM Serving Infrastructure

Multi-provider LLM gateways reverse proxies that route, load-balance, and rate-limit requests across foundation-model APIs have become critical production infrastructure. Yet the failure modes specific to this architectural layer remain undocumented, scattered across issue trackers and post-mortems with no unifying framework. We introduce \fa{}, a two-axis taxonomy that classifies failures by their \emph{origin layer} (Network/Transport, Streaming/Protocol, State/Session, Model~Behavior, Governance/Cost) and their \emph{detectability} (Loud vs.\ Silent). We populate this taxonomy with five verified catalog entries sourced from public bug reports and first-hand stress testing, each accompanied by a mechanistic root-cause analysis. Three entries include standalone reproduction scripts. Our principal finding is that the most operationally severe failures are \emph{silent}: they return HTTP~200, pass every standard health check, and corrupt application state in ways that require semantic-level observability to detect. Two such silent failures a concurrency race condition causing history loss and a streaming index collision corrupting tool-call payloads were discovered first-hand during \cb{} evaluation campaigns.

HyNA: Taming Tail Latency in MoE Training with Hybrid Switch Silicon

The transition to trillion-parameter models, particularly Mixture-of-Experts (MoE), shifts the bottleneck of distributed training from computation to communication. However, existing Parameter Server (PS) architectures succumb to incast congestion, while state-of-the-art In-Network Aggregation (INA) solutions like ATP fail to handle the sparse, bursty traffic of MoE workloads. These solutions suffer from severe tail latency amplification due to their reliance on slow, host-based fallbacks for collisions and overflows. To dismantle this communication wall, we propose HyNA, a fully serverless aggregation system that eliminates dedicated parameter-server nodes by leveraging a novel hardware-software co-designed switch architecture. HyNA couples wire-speed Reconfigurable Match Tables (RMT) with embedded RISC-V cores. By adhering to a strict on-chip closure principle, the system processes all traffic anomalies—including hash collisions and floating-point variances—entirely within the switch ASIC, converting unpredictable network RTT into deterministic on-chip latency. We validate our design through a 100 Gbps FPGA prototype and a 7nm ASIC synthesis analysis. Results demonstrate that HyNA incurs less than 3% silicon area overhead while improving aggregation throughput by 7.35X over BytePS and 1.4X over ATP. Crucially, in the MoE gradient synchronization phase, the system eliminates the fallback penalty and reduces synchronization time by up to 1.6X compared to dynamic INA baselines, without compromising bit-level model accuracy.

Finding NEMO: Nimble and Expressive Memory Observability

CCSwitch: A Scalable Data Plane for Non-Blocking In-Network Collective Communication

This work presents CCSwitch, a modular switching fabric built from 4×4 non-blocking Collective Engines, a modular switching fabric built from 4×4 non-blocking Collective Engines (CEs) that combines spatial and temporal parallelism to perform reductions without accumulation buffers.

Delphinus: Ultra-Fast Link Failure Detection and Recovery for AI Data Center Networks

High-performance artificial intelligence (AI) applications impose stringent reliability requirements on AI data center networks (DCNs), yet link failures are almost inevitable and can severely disrupt AI workloads such as large language model (LLM) training and inference. Existing deployed link failure detection and recovery mechanisms suffer from slow execution speed and limited failure coverage, failing to meet the demands of production AI DCNs. To address these issues, we propose Delphinus, an ultra-fast link failure detection and recovery solution built on the data-plane of programmable switches. It achieves ultra-fast failure detection via hardware-based port state monitoring, extends recoverable failure coverage through remote failure notification and relay, and enables fast recovery by path switchover. Delphinus can serve as a key generic function of switches, providing host-transparent link failure handling for Ethernet fabrics. We implement Delphinus on commercial hardware switches, and deploy it in large-scale production AI DCNs for over a year. Extensive evaluations demonstrate that Delphinus can complete link failure detection and recovery within sub-milliseconds, with negligible impact on application performance and imperceptible service interruption.