Noam Razin

Understanding Deep Learning via Notions of Rank

It is established that gradient-based training can induce an implicit regularization towards low rank for several neural network architectures, and it is demonstrated empirically that this phenomenon may facilitate an explanation of generalization over natural data.