This work formalizes learnability as the success rate of converging to the global minimum, as a function of overparameterization, learning algorithm, student initialization distribution, and teacher geometry, and shows that differentially increasing the learning rate of the readout layer and decreasing the learning rat...
Kai J. Sandbrink, Flavio Martinelli, Alexander van Meegen et al.· 0 citations
DiG-bench (Discovery in Games) is released, a new benchmark for discovering new knowledge with experimentation in controlled environments where the objective is unknown, which provides games at seven tiers of difficulty for AI agents.
R. Battleday, Kai J. Sandbrink, Jimi Cullen-Drohan et al.· 2 citations· ⚡1
It is shown that imbalanced learning of two conflicting copy tasks promotes in-context learning and improves the selectivity of refusal fine-tuning, suggesting that imbalanced pretraining curricula may be an important tool for promoting disentangled representations, with direct consequences for the precision and reliab...
Sebastian A. Bruijns, Jirko Rubruck, Mia Whitefield et al.· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.