Optimizer Memory Schedules for Outscaling the Overtraining Axis
The study of how matrix-preconditioned methods and a momentum-scheduled method scale relative to AdamW establish training horizon as an essential axis for optimizer evaluation and design.
Katie Everett, Shi-Kai Qiu
· 1 citation