Min Chen

Scaling and Renormalization in Neural Networks: An Effective Renormalization-Group Description of Grokking

Grokking is a delayed transition from memorization to generalization in deep neural networks (NNs). We examine whether scaling and renormalization group (RG) theories can describe this dynamics, establishing a quantitative physical framework for neural generalization transitions. For modular addition on a two-layer NN,...