HBF memory system HBFlex, a full-HBF memory system with coordinated optimizations for KV reads, writes, and reclamation, is presented, benefiting from higher HBF bandwidth and more efficient management of dynamic KV-cache reads, writes, and erases.
Shu-Zhang Zhong, Wei-Kai Xu, Yi-Fan Zhou et al.· 1 citation
HBF sucks as an SSD replacement for transient KV, but earns its place in LLM serving when used selectively with reuse-aware placement, write budgeting, and thermal coordination.
Zhuo-Ran Li, Zhuo-Hang Bian, Xin Huang et al.· 5 citations
HDA-MoE is presented, a framework that optimizes MoE execution on NMP architectures through hybrid parallel deployment and runtime scheduling and integrates an offline hybrid parallel mapping algorithm with an online dynamic and adaptive scheduling mechanism to reduce communication overhead while improving computation...
Hao-Chen Huang, Shu-Zhang Zhong, Sheng-Xuan Qiu et al.· IEEE Transactions on Compute...· 0 citations
CODA is proposed, an algorithm-hardware co-designed architecture centered on Compute-Cache Operator Disaggregation that achieves up to 1.80x end-to-end speedup and 1.74x higher energy efficiency, while preserving competitive generation quality compared with a state-of-the-art caching algorithm.
