Momentum as Residual-Driven Multiplier Correction for Deep Learning Optimization
An AIM framework based on residual-penalty variable splitting, which interprets momentum as a multiplier-like correction driven by the splitting residual, and RADAR, which combines relativistic adaptive geometry, decoupled residual correction, and second-order momentum filtering to improve the update direction and momentum estimation.
Zhi-Xin Ren, Yao Lyu, Congrong Li et al.
· 0 citations