2026· IEEE Transactions on Networking· Vol 34, pp. 6678-6692· 0 citations· 45 references
Abstract
Due to the high operational complexity and limited deployment scale of lossless RDMA networks, the community has been exploring efficient RDMA communication over lossy fabrics. State-of-the-art (SOTA) lossy RDMA solutions implement a simplified selective repeat mechanism in RDMA NICs (RNICs) to enhance loss recovery efficiency. However, these solutions still face performance challenges, such as unavoidable ECMP hash collisions and excessive retransmission timeouts (RTOs). In this paper, we enhance RDMA reliability with the goals of being independent of PFC, compatible with packet-level load balancing, free from RTO, and friendly to hardware offloading. To this end, we propose DCP, a transport architecture that co-designs both the switch and RNICs, fully meeting the design goals. At its core, DCP-Switch introduces a simple yet effective lossless control plane, which is leveraged by DCP-RNIC to enhance reliability support for high-speed lossy fabrics, primarily including header-only-based retransmission and bitmap-free packet tracking. We prototype DCP-Switch using P4 switch and DCP-RNIC using FPGA. Extensive experiments demonstrate that DCP achieves $1.6\times $ and $2.1\times $ performance improvements, compared to SOTA lossless and lossy RDMA solutions, respectively.
ProLet is a load balancing scheme that enables proactive probing and reroutes elephant flows at flowlet granularity in lossless RDMA networks and reduces average and tail flow completion time slowdowns by 69% and 79%, respectively, compared to state-of-the-art load balancing schemes.
Hong Wang, Jin-Hao Luo, J. Tan et al.· Asia-Pacific Workshop on Net...· 0 citations
Existing RDMA transports for AI fabrics compromise on scalability, robustness, or hardware overhead. Lossless protocols suffer from PFC-induced congestion spreading under extreme incast, while switch-assisted lossy transports collapse into catastrophic Retransmission Timeout (RTO) storms under silent physical link errors. Furthermore, supporting packet-level load balancing typically requires prohibitive SRAM for out-of-order tracking. We present STAR, a purely end-to-end, stateless, and receiver-driven RDMA architecture. STAR employs chunk-level token pacing to temporally interleave concurrent flows, intrinsically suppressing incast queues and eliminating PFC dependency. By extending Direct Data Placement (DDP) semantics for idempotent memory scatter, it absorbs out-of-order packets with $\mathcal{O}(1)$ hardware state. To handle rare physical drops, an app-aware software watchdog performs full-chunk retransmission, circumventing hardware RTO storms. Evaluations demonstrate that STAR scales linearly under extreme 63-to-1 incast, introduces negligible (~2.5%) collective completion time overhead in ideal lossless fabrics, and prevents catastrophic synchronization barrier stalling under physical link errors.
Jing Liu, NingCheng Wang, Lun-Sheng Li et al.· Fall Joint Computer Conferen...· 0 citations
Remote Direct Memory Access (RDMA) is widely adopted for high-performance datacenter workloads, but its strict ordering requirement conflicts with multipath transport in lossy networks. RDMA Network Interface Cards (RNICs) may interpret Out-of-Order (OOO) arrivals caused by diverse path latencies as packet loss. This triggers unnecessary retransmissions, severely degrading goodput collapse and inflating Flow Completion Time (FCT). We present PSN-PATH, a novel RDMA protocol to distinguish packet loss from OOO and avoid unnecessary retransmission, improving goodput and FCT of multi-path RDMA in lossy networks. PSN-PATH maps Packet Sequence Numbers (PSN) to network paths, enabling the receiver to verify packet continuity within each mapped path and distinguish real losses from OOO arrivals. PSN-PATH only retransmits lost packets and redirects traffic away from degraded paths. We implement PSN-PATH on Xilinx Alveo U50 FPGA using Vitis HLS and validate it via extensive simulation experiments. Experimental results demonstrate that PSN-PATH significantly outperforms existing solutions, reducing FCT by 95.81%, enhancing goodput by 5.11×, and reducing retransmissions by over 99% under high-load, lossy conditions.
Zhexiong Li, Shugui Wei, Puyu Zhao et al.· Conference on Applications,...· 0 citations
Providing secure and reliable end-to-end connectivity for distributed applications in wide area networks remains a major challenge. However, the support for reliability mechanisms in common virtual private network (VPN) solutions is limited. In our work, we investigate retransmissions, forward error correction (FEC), multipath connections, and overlay networks to improve VPN reliability. In this context, we showcase the specific potentials and limitations of these mechanisms. Our MASQUE-based VPN prototype, implements these reliability measures. We conduct a measurement campaign to assess the behavior of various configurations under different path impairments. As a baseline comparison, we include the common VPN solutions WireGuard and OpenVPN and evaluate throughput, end-to-end loss, and one-way delay. MASQUE uses congestion control also in the unreliable mode, which can lead to problems when the available bandwidth is misjudged. Retransmissions eliminate end-to-end loss but introduce significant long-tail delay. With FEC-based recovery, the long-tail delay is decreased. Multipath VPNs improve resilience in deteriorated network conditions. Further, we nest multipath MASQUE connections and form overlay networks. They increase path diversity, and, hence, circumvent unreliable path segments.
Kilian Holzinger, Lion Steger, Michael Haden et al.· Applied Networking Research...· 0 citations
As RDMA becomes increasingly tolerant to out-of-order delivery, fine-grained packet-level multipathing is emerging as a practical design for datacenter fabrics. Packet spraying and related schemes improve load distribution for large flows, but they also force RDMA flows onto multiple paths whose conditions can differ at short timescales due to randomized traffic placement. For short flows, even one packet sent on a temporarily slower path can delay the entire flow. As a result, fine-grained load balancing can hurt, rather than help, small multi-packet flows. We present PIN, a lightweight wrapper that layers on top of any existing fine-grained RDMA load balancer. PIN applies less multipathing to short flows, which benefit from it least: it pins short flows to one path for their lifetime and leaves larger flows to the baseline mechanism. The design is simple, compositional, and requires only a size threshold. We show analytically that practical thresholds preserve overall load-balance fairness while reducing short-flow exposure to temporarily slower paths. Large-scale simulations across a variety of workloads show that PIN consistently improves both mean and tail completion time for short flows across multiple RDMA load-balancing baselines.
Jichun Wu, Ran Shu, A. Moore et al.· Asia-Pacific Workshop on Net...· 0 citations
This work introduces CSIG, a protocol that delivers precise, multi-bit bottleneck congestion signals via a fixed-length Ethernet header, and proposes Fast Ramp-Up, a congestion control primitive that leverages these bottleneck signals to reduce median RPC latency by 20% and unclaimed bandwidth by 60% in production.
Abhiram Ravi, Nandita Dukkipati, Weiwu Pang et al.· Conference on Applications,...· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.
