Modern HPC systems increasingly rely on tiered storage architectures with SSDs serving as a critical performance tier. However, the inherent asynchronous I/O characteristics of SSDs, including read/write bandwidth asymmetry and interference, pose significant challenges for traditional I/O schedulers. These challenges are exacerbated by the convergence of bursty HPC write workloads (e.g., checkpointing) and sustained AI read workloads (e.g., data streaming) on shared SSD infrastructure. Existing schedulers fail to adequately address these combined workloads, leading to suboptimal resource utilization. This paper introduces HARMONI, a heterogeneity-aware reinforcement learning scheduler for mixed I/O in HPC storage systems. HARMONI leverages a graph neural network (GNN) to encode task-SSD dependencies and a hybrid interference predictor to adapt to hardware and I/O variations. Experimental results across diverse HPC and AI workloads demonstrate that HARMONI significantly reduces average makespan by up to 90% compared to state-of-the-art schedulers, effectively bridging the gap between evolving storage hardware and the dynamic I/O demands of modern HPC systems.
Ze-Xi Cai, Tong Zhao, Shadi Ibrahim et al.· ACM Transactions on Architec...· 0 citations
Graph-based Approximate Nearest Neighbor Search (ANNS) has become fundamental to modern data-intensive applications, yet supporting efficient vector updates while maintaining index quality remains a critical challenge. Existing approaches face a tough dilemma: extensive reconstruction ensures connectivity but incurs unpredictable overhead, while restricting update scope improves efficiency but degrades search accuracy. In this paper, we reveal that graph-based ANNS indexes inherently comprise backbone nodes that dictate global navigation capability and expansion nodes that provide local refinement. This insight motivates Hope, a Host-CSD (Computational Storage Device) cooptimized update framework that optimizes both index quality and update efficiency with a differentiated update design. For efficient vector categorization, we propose an in-storage sketcher that leverages per-dimension correlation with the bitmap-based flip mechanism to dynamically identify backbone and expansion vectors with minimal overhead. For asymmetric update handling, Hope employs the host-CPU for computation-intensive backbone updates with multi-hop neighbor gathering to preserve connectivity, while delegating I/O-intensive expansion updates to in-storage computing for immediate local manipulations. Our evaluation demonstrates that Hope achieves superior update efficiency and index quality compared to state-of-the-art baselines, providing stable performance for both vector updates and ANNS queries in dynamic environments.
Kecheng Huang, Jiaxian Chen, Chenlin Ma et al.· IEEE Non-Volatile Memory Sys...· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.
