Jul 2026
Hyperball May Not Be a Free Lunch
An angular effective learning rate is derived that accounts for the parameter-update angle, parameter norm, and update norm, and shows that the conventional norm-based measure is a special case under parameter-update orthogonality.
Yihao Xiao, Jialong Sun, Zitian Gao et al.
· arXiv.org · 3 citations