LocalProp: Neuro-Localized Memory-Efficient Backpropagation
This work proposes LocalProp, a training procedure that locally updates the weights of a model, where the pre-training is based on I-JEPA, and empirically shows that it reaches good performance at a fraction of GPU peak memory.
Diana-Nicoleta Grigore, Iuliana Georgescu, R. Ionescu
· 0 citations