Category
2,791 papers
This monograph presents a unified statistical framework for reliable post-training hyperparameter selection, centered on the learn-then-test (LTT) paradigm, that enables the choice of hyperparameters that provably satisfy application-specific reliability requirements.
Gaussian process (GP) predictive distributions are commonly used in Bayesian optimization (BO) to guide the selection of evaluation points for expensive objective functions. The choice of kernel and hyperparameters has a strong influence on the exploration--exploitation trade-off. For minimization, sampling criteria su...
The first scaling law for continuous DLMs that rivals discrete DLMs is established: RePlaid exhibits a compute gap of only $20\times$ compared to autoregressive models, outperforms Duo while using fewer parameters, and outperforms MDLM in the over-trained regime.
In contextual bilateral trade under full feedback, the posted price does not affect which valuations are observed. We show that in this model such action-independent feedback removes the polynomial adaptation penalty familiar from heavy-tailed bandits: fully parameter-free algorithms attain the oracle minimax $T$-expon...
Reach audiences
Advertise in front of researchers, engineers, and readers.
A statistical framework for Inverse Entropy-regularized Reinforcement Learning is developed that resolves ambiguity by combining entropy regularization with a least-squares reconstruction of the reward from the soft Bellman residual, leading to a unique and well-defined so-called least-squares reward consistent with th...
The theory reveals that TTT enables the single-layer transformers to adapt to both the feature vector and the link function, which vary across tasks, which creates a sharp contrast with ICL alone, which is theoretically difficult to adapt to shifts in the link function.
The analysis is extended to networks with possibly non-zero biases drawn from arbitrary symmetric distributions and derives high probability upper and lower bounds for wide networks that differ at most by a factor that is logarithmic in the network's width and linear in its depth.
For two causal structures with the same set of visible variables, one is said to observationally dominate the other if the set of distributions over the visible variables realizable by the first contains the set of distributions over the visible variables realizable by the second. Knowing such dominance relations is us...
Gaussian Mixture Models (GMMs) range among the most frequently used models in machine learning. However, training large, general GMMs becomes computationally prohibitive for data sets that have many data points $N$ of high-dimensionality $D$. For GMMs with arbitrary covariances, we here derive a highly efficient variat...
This work studies Fisher-Rao gradient flows of linear programs more generally and shows linear convergence with a rate that depends on the geometry of the linear program, and shows sublinear convergence for perturbed Fisher-Rao gradient flows and natural gradient flows up to an approximation error.
Understanding how neural networks learn and organize features is central to understanding their behavior. Much existing theory of feature learning has focused on the emergence of a global low-dimensional representation. We show that this picture is incomplete. In regression problems with clustered data, we demonstrate...
This work proves the existence of an optimal hyperparameter configuration for which the LGD algorithm achieves the Bayes'optimal solution for squared loss on regression tasks, and shows empirical evidence of the success of LGD and the meta learning procedure for few-shot learning on linear and logistic regression using...
General Science
General Science
Learn how Google DeepMind and GovTech Singapore uses lightweight on-device Gemma models to intercept conversational scams in private chats with zero cloud data egress.
MIT Statistics and Data Science Center Director Alexander (Sasha) Rakhlin shares important considerations for departments and institutions.
MIT Statistics and Data Science Center Director Alexander (Sasha) Rakhlin shares important considerations for departments and institutions.
MIT Statistics and Data Science Center Director Alexander (Sasha) Rakhlin shares important considerations for departments and institutions.
We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.
