Sparse Weight Decomposition is proposed, which reparameterizes pretrained linear projections by factorizing each weight matrix into two sparse factors whose shared intermediate coordinates serve as individually addressable circuit units.
Chuanhao Yan, Xuhan Huang, Yawen Duan et al.· 1 citation
An angular effective learning rate is derived that accounts for the parameter-update angle, parameter norm, and update norm, and shows that the conventional norm-based measure is a special case under parameter-update orthogonality.
It is found that OPD transfers a teacher's reasoning behavior rather than its answers to particular problems: training difficulty barely matters, and even problems the teacher never solves are useful.
Zhaoyi Li, Deyang Kong, Yuan Wei et al.· 1 citation
This work presents a unified black-box RL framework for stable and scalable optimization of general agents through complex harnesses, supporting unified training across heterogeneous execution systems.
Hua-Tong Song, Fei Bai, Ming Yang et al.· 2 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.