Aug 2026· IEEE Non-Volatile Memory System and Applications Symposium· pp. 1-6· 0 citations· 25 references
Abstract
Erasure coding provides efficient fault tolerance for large-scale distributed storage systems. However, its data repair process is well-known to be resource-intensive. We find that conventional host-centric, TCP-based repair architectures suffer from severe resource contention. Even in high-bandwidth networks, such interference slows repair operations and degrades service quality. To address this limitation, we propose DPUEC, a novel architecture that fully offloads the entire repair process to a Data Processing Unit (DPU). DPUEC leverages Remote Direct Memory Access (RDMA) to enable kernel-bypass data transfers and utilizes the onboard multi-core processors of DPU for highly parallel decoding. This design establishes a physically isolated data plane for repair traffic, completely bypassing host resources. Our prototype evaluation shows that DPUEC improves repair throughput by 133% - 181% over the state-of-the-art approach across different workloads. More importantly, DPUEC delivers true performance isolation. It eliminates any interference with foreground applications and reduces host CPU utilization on the recovery node to nearly zero.
Spillway introduces a DPU-host hybrid data plane that repurposes idle host CPU resources to process spillover traffic when the DPU becomes the bottleneck, and decouples virtual switching capacity from static DPU hardware limits.
Xiaochong Jiang, Dian Fan, Yilong Lv et al.· Conference on Applications,...· 0 citations
Erasure coding is widely used in distributed storage systems because it provides high reliability with low storage overhead. To adapt to changing performance and reliability requirements, systems may perform redundancy transition on erasure-coded data, i.e., re-encoding existing data on the fly with new coding parameters. However, existing random data layouts incur substantial transition traffic and severe load imbalance. We present DdlRT, a deterministic data layout for redundancy transition in distributed storage systems. DdlRT leverages two orthogonal arrays to build a two-level mapping across racks and nodes, deterministically placing stripes on designated storage nodes. By making data layout transition-aware, DdlRT avoids or significantly reduces transition traffic while preserving load balance at both the rack and node levels. Large-scale simulations and real-system experiments show that, compared with state-of-the-art schemes, DdlRT reduces redundancy transition traffic by 68.13%–92.37% and transition time by 38.32%–94.35%, while also improving load balance.
Xiao-Tong Geng, Hao-Qi Duan, Liangliang Xu et al.· Proceedings of the Internati...· 0 citations
Efficient data movement is a critical bottleneck in High Performance Computing (HPC) centers, especially for large-scale migrations across parallel and distributed storage systems. Existing tools such as rsync and rclone often underutilize network and metadata bandwidth, provide limited hard link preservation, and offer no guarantees on memory usage, limiting their effectiveness for HPC environments. Motivated by a large-scale migration of data at Partnership for an Advanced Computing Environment (PACE), we present copy2, a high-performance, resource-aware data transfer tool for modern HPC storage such as Lustre and VAST Data. copy2 features a scalable metadata crawler, inode-based hard link preservation, asynchronous high-throughput transfers, and bounded memory allocation. Preliminary results show up to twice the throughput of rclone with substantially lower energy usage, and successful migration of projects with hundreds of millions of files that would be impractical with existing tools. copy2 offers HPC administrators and researchers a robust, predictable, and efficient solution for large-scale data management.
A. Lambert, Deepa Panish· Practice and Experience in A...· 0 citations
OpScale is presented, a practical operator-level orchestration framework of profiling, provisioning, placement, and runtime serving that attains SLOs with up to 36.3% fewer GPUs and 28% less power, or achieves 44% higher throughput under fixed cost budgets.
Xingqi Cui, Chieh-Jan Mike Liang, Ziang T. Tang et al.· 0 citations
Rack-scale interconnects serve as critical datapaths for emerging communication-intensive systems to scale up. Innovative solutions for this datapath are rising at a rapid pace, especially those based on Ethernet. However, existing hardware-based solutions, such as RDMA, face performance issues, particularly for small-message memory access, and suffer from the inflexibility of hardware-fixed processing. The community is actively pursuing efficient, flexible, and cost-effective rack-scale datapaths. In this work, we propose Software-Interposed Datapath (SID), an efficient, software-flexible, and low-cost solution for rack-scale interconnects, particularly optimized for fine-grained memory access. Improving small-message efficiency is a well-known challenge, and software involvement for flexibility seems to amplify the performance hurdle further. SID boosts performance by exploiting one insight: existing NICs primarily rely on Queue Pair (QP)-level parallelism, but underutilize intra-QP Work Queue Element (WQE)-level parallelism. Harnessing parallelism is non-trivial, especially at the WQE-level, due to ordering semantics and request dispatching. The key technique is our Elastic QP data structure built atop the on-NIC datapath processors, which realizes ordered intra-QP parallelism while minimizing coordination overhead. Regarding flexibility, SID supports extensible operation sets that comply with the OpenSHMEM model for ML/HPC workloads and the Message Queue model for cloud service workloads. Regarding cost efficiency, SID is built on top of commodity components such as Ethernet, PCIe, and datapath cores of NVIDIA ConnectX-8 and BlueField-3 NICs. Evaluation shows that SID achieves up to 11.03x higher rates for small messages than RDMA-based baselines and supports both CPU and GPU-Direct operations.
Chenxingyu Zhao, Yibo Wu, Hongtao Zhang et al.· Conference on Applications,...· 0 citations
Processing-in-Memory (PIM) promises to reduce data movement overhead by executing computation in or near memory, but its realized application speedup remains highly design-dependent. Non-offloadable host execution, host-PIM transfers, limited PIM capacity, and device programming latency can limit end-to-end speedup, making fast early-stage design-space exploration (DSE) essential. However, existing PIM evaluation methods remain limited: circuit- and device-level tools cannot capture these end-to-end PIM performance factors, while cycle-accurate simulation is too slow for iterative DSE. To address this gap, we present VIPER, a unified, lightweight, and architecture-aware performance evaluation framework for PIM DSE. VIPER profiles host execution once and combines the measured host behavior with a PIM-aware analytical engine that sweeps PIM-side parameters across candidate designs. It supports both Processing Near Memory (PNM) and Processing Using Memory (PUM) under task-offloading and data-triggered execution by capturing host-PIM transfer, array access, in-memory computation, device programming latency, and capacity-induced partitioning, providing rapid architecture-aware performance estimates for iterative DSE without repeated cycle-accurate simulation. We validate VIPER against a commercial UPMEM system and more than 400 cycle-accurate gem5 configurations. VIPER predicts the UPMEM offloading decision and break-even region a priori, and, with a refined transfer model, captures the measured peak-and-rolloff behavior with 12\% mean speedup error across the DPU sweep (6\% up to the 256-DPU peak). Against gem5, VIPER achieves less than 10\% error while reducing evaluation time from hours to under one minute. Case studies of UPMEM, ReRAM/FeFET crossbars, and IMCRYPTO show that architecture-aware DSE reveals key performance trade-offs that device-level evaluation misses.
Haoran Geng, T. Pereira, Xiaoyang Lu et al.· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.
