The theoretical analysis of SFS reveals that SFS exhibits behavior closer to IRRI than to Line, and these results suggest that a simpler method, IRRI, may achieve a similar refinement process without relying on the complex correction mechanism of SFS.
This work revisits classical sentence classification models from a margin enlargement perspective, aggregating token-level features via max pooling and directly estimating sentence scores using a lightweight MLP, and achieves substantial efficiency improvements while maintaining competitive performance with state-of-th...
Practical guidelines for applying randomized Hadamard rotation, sign selection, and channel scaling are derived, which explain how random signs suppress constructive interference among persistent outlier channels, how sampling multiple sign patterns can improve transformation selection, and how second-moment balancing...
This study provides a theoretical analysis showing that gradient heterogeneity, together with Hessian heterogeneity, degrades the convergence of gradient-based methods such as SGD, while sign-based methods are substantially less sensitive to this effect.
Akiyoshi Tomihari, Issei Sato· 6 citations· ⚡1
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.