Towards Easier-to-Tune Training Algorithms: New Results from AlgoPerf
Evaluating deep learning optimizer is challenging because confounding factors such as hyperparameter tuning and optimizer-model co-design often obscure true algorithmic performance. We present results of the updated AlgoPerf V1 benchmark, in which we evaluate a new set of training algorithms on wall-clock time-to-targe...
Priya Kasimbeg, Ahmed Khaled, Aahlad Chandrabhatta et al.
· 0 citations