Analytically, the spectrum is solved in the lazy high-dimensional limit for both linear ($n \asymp d) and polynomial ($n \asymp d^k$) sample complexities, and it is proved through a bias--variance decomposition that the first bulk minimizes the approximation error while the second drives the error associated with memor...
Raphaël Urfin, Tony Bonnaire, Giulio Biroli et al.· arXiv.org· 1 citation
Score-based generative models generate new samples by integrating a time-dependent drift that carries Gaussian noise onto the target distribution. In practice this drift is modeled by a neural network, trained on a loss integrated over time $t$ with a weighting schedule $w(t)$. Along the backward dynamics, and for mult...
J. Klinger, Raphaël Urfin, Giulio Biroli et al.· 0 citations
Overparameterization remains beneficial when paired with regularization: in the random-features theory and in U-Net experiments, optimally regularized large models---via a ridge penalty or early stopping, respectively---outperform any unregularized models.
Raphaël Urfin, Tony Bonnaire, Giulio Biroli et al.· arXiv.org· 0 citations
It is found that τmem increases linearly with the training set size n, whereas τgen remains constant, which creates a growing window of training times with n where models generalize effectively, despite showing strong memorization if training continues beyond it.
Tony Bonnaire, Raphaël Urfin, Giulio Biroli et al.· Journal of Statistical Mecha...· 2 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.