HBF memory system HBFlex, a full-HBF memory system with coordinated optimizations for KV reads, writes, and reclamation, is presented, benefiting from higher HBF bandwidth and more efficient management of dynamic KV-cache reads, writes, and erases.
Shu-Zhang Zhong, Wei-Kai Xu, Yi-Fan Zhou et al.· 1 citation
NAMOH, an architecture-native sparse attention mechanism that activates only its assigned tokens and performs causal attention within this subsequence, is introduced, and it is hoped this work offers a new path for scaling attention, with parameter scaling directly enabling context scaling.
HDA-MoE is presented, a framework that optimizes MoE execution on NMP architectures through hybrid parallel deployment and runtime scheduling and integrates an offline hybrid parallel mapping algorithm with an online dynamic and adaptive scheduling mechanism to reduce communication overhead while improving computation...
Hao-Chen Huang, Shu-Zhang Zhong, Sheng-Xuan Qiu et al.· IEEE Transactions on Compute...· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.