Orthogonal Witness Control for Muon Optimization via Sigmoid Spectral Reshaping
A matrix-valued optimizer that preserves the singular subspaces of the gradient while applying a bounded, monotone sigmoid transformation to its singular values is introduced, and the effectiveness and robustness of Soren against established optimizers are demonstrated.