Critical initialization destabilizes higher input derivatives in wide scalar-input networks
It is proved that every fixed finite derivative order has uniformly bounded variance under explicit regularity assumptions, and for residual networks with branch scale L^{-1/2}, it is proved that every fixed finite derivative order has uniformly bounded variance under explicit regularity assumptions.
Prashant Singh, Pranav Singh
· 0 citations