Fabian Zehetgruber

High Probability Derivative Bounds for Random tanh Neural Networks on a Hypercube

We establish high-probability bounds for mixed input derivatives of wide random neural networks whose activation derivatives satisfy a factorial growth bound. Our main result specializes these estimates to $\tanh$ networks with Xavier initialization. A direct deterministic analysis based on Euclidean operator norms of...