Author

Aditya Somasundaram

3 papers indexed here

We haven’t gathered this author’s papers yet. Follow them and we’ll fetch their work.

Not the right person? Other researchers publish under this name.

#artificial intelligence Preprint Oct 2026

Taming Softmax at the Limits of Muon: A Unified Approach through Hilbert Geometry

Muon controls weight updates through the RMS movement of a linear layer's output. This implicitly treats the linear output as a proxy for the full module, including the nonlinearity that follows. This proxy can be misleading for softmax, where the RMS shift in activations does not faithfully measure the change in the o...

Aditya Somasundaram, Charles Guille-Escuret, Alexander Moreno et al. · 0 citations

Preprint Aug 2026

Toward a First-Principles Update Geometry for the Language-Model Head

Muon motivates designing optimizer geometry around the function of each parameter block and uses the spectral norm for hidden linear layers. For the language-model head, the spectral norm is not a faithful measure of functional change. Softmax removes shared logit shifts, whereas the spectral norm can assign arbitraril...

Aditya Somasundaram, Charles Guille-Escuret, Alexander Moreno et al. · 0 citations

Preprint Aug 2026

Toward a First-Principles Update Geometry for the Language-Model Head

Motivated by Muon’s singular-value conditioning, this work proposes maximizing the smallest row separation while constraining this diameter, producing an approximate-equidistance problem when V ≫ d .

Aditya Somasundaram · 0 citations