BASIC-Prefetcher: Bin-based Address and Size-Informed Caching for AI-Driven SSD Workloads
TL;DR
The BASIC Prefetcher is proposed, which recovers application-level I/O patterns by classifying requests into size-based bins and modeling temporal bin transitions with a lightweight Markov-chain framework and achieves up to 89% prediction accuracy on sequential workloads and 70% on highly random workloads, including AI large language model inference traces.
Abstract
Read latency is a critical bottleneck for NAND-based SSDs in AI-driven datacenter workloads, where model parameters and key-value data are frequently swapped between main memory and storage. Existing prefetching schemes operate on block-level address sequences that have been stripped of application context by the filesystem layer, limiting their prediction accuracy and coverage. We propose the BASIC Prefetcher, which recovers application-level I/O patterns by classifying requests into size-based bins and modeling temporal bin transitions with a lightweight Markov-chain framework. By confining address prediction to frequently accessed bins and their primary data regions, BASIC Prefetcher achieves up to 89% prediction accuracy on sequential workloads and 70% on highly random workloads, including AI large language model inference traces, without requiring pre-training or per-page state tracking. Across six diverse workloads, our approach improves SSD throughput by 1.41× to 2.39× over existing prefetching schemes.