Fisher information and the geometry of memorization in neural networks
A fruitful approach towards understanding generalization in machine learning involves characterizing how models separate generalizable structure from idiosyncratic or even corrupted features of training data. Fisher information has emerged as a diagnostic tool for this separation, leveraging loss geometry to distinguis...
Daniel T. Bernstein, Chase Goddard, L. di Carlo et al.
· 0 citations