Hot–Cold Tiering of HBM and High Bandwidth Flash for Agentic LLM Serving
This work places the hot set in HBM and the cold pool in HBF, forming a hot-cold KV hierarchy within the GPU memory tier, while flash-establishing HBF as a cold-tier complement to HBM rather than its replacement.
Jongjin Baek, Won Ji, Seungjae Yoo et al.
· IEEE computer architecture l... · 0 citations