This paper characterise the optimal split under data-dependent evaluation geometries induced by the fine-tuning problem as a compute-split problem under a two-stage pretrain--fine-tune procedure with fixed total optimisation budget, using regularised least squares trained by gradient descent as a tractable setting.
Alex Buna, Fang-Hui Liu, Patrick Rebeschini· 0 citations
This work studies regression under bounded responses in terms of excess mean squared error and proves several additional structural results about universal rates in learning with squared loss about minimax and universal exponential rates.
M. Høgsgaard, Patrick Rebeschini, Tobias Wegel· 0 citations
This work shows that early stopping can overcome suboptimality: in a Gaussian mixture model with label-flipping noise, GD stopped at an appropriate oracle time achieves minimax-optimal excess zero-one risk for covariance spectra with fast and continuous decay, including polynomial and exponential spectral decays.
Alex Buna, S. Liu, Patrick Rebeschini· 0 citations
The aggregation with exponential weights (AEW) estimator is not fully understood in the basic setting of model selection aggregation with squared loss. In particular, whether it is minimax-rate optimal in expectation for large enough fixed temperatures and under random design has been an open problem since its introduc...
M. Hogsgaard, Patrick Rebeschini, Tobias Wegel· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.