A Flatness-Generalization Relation in the Teacher-Student Tree-Committee Machine
The flatness of the loss landscape at a minimizer is a widely used heuristic for reasoning about neural-network generalization, yet evidence for this relation is mostly empirical and controversial. We study this relation in a teacher-student tree committee machine, where both the ERM estimator and the Hessian spectrum...
B. Annesi, Davide Straziota, E. M. Malatesta
· 0 citations