Jonghyeon Nam

Fusing the Attention Training Dataflow with Local Safe Softmax and Model-Independent Tiling

Software simulations and hardware evaluation show that the proposed operation fusion for the attention training dataflow using local safe softmax and model-independent tiling reduces off-chip access, on-chip memory usage, FLOPs, training runtime, and energy cost compared to conventional approaches, confirming its suita...