Fast Learning Rate Transfer in Shallow Linear Networks at Growing Training Horizons
This work investigates conditions that ensure fast transfer in the growing-horizon regime of hyperparameter transfer in a shallow linear network with a single trainable hidden matrix, trained by full-batch gradient descent.
Mana Sakai, Masaaki Imaizumi
· 0 citations