It is suggested that simple in-training interpolation deserves consideration before introducing more elaborate forgetting-mitigation mechanisms, and Sequential Fine-tuning with Averaging with Averaging with SFA is introduced and evaluated.
Anat Kleiman, G. Dziugaite, Jonathan Frankle et al.· 16 citations· ⚡2
Vision-language reward models are trained on annotated preference pairs, which are expensive to collect, can disagree, and are ranked by benchmark accuracy. We build one with no human preference label: a linear head on an intermediate layer of a frozen Qwen3-VL backbone, trained to score a web caption above a perturbed...
Yue-Ying Li, Jing-Xuan Fan, Kianté Brantley et al.· 0 citations
This work studies blackboard intelligence: an inference-time perspective in which a model works on a fixed, revisable canvas and searches over candidate solution states rather than committing to a causal, left-to-right trajectory, and concludes that mean confidence, a simple model-internal quantity available from the s...
Woosang Jeon, Jaeyeon Kim, S. Kakade et al.· 0 citations
The theoretical results show that GN's optimality for linear regression no longer holds under a poorly chosen basis under both population and stochastic updates, or a move from linear regression to a non-convex variant of logistic regression, and Adam-style methods offer genuine advantages over curvature-inspired preco...
Due to the complexity of neural network loss landscapes, optimization theory is forced to rely on idealized models, and there is generally a tradeoff between how theoretically tractable the model is, and how accurately it describes the true optimization dynamics. In this work, we stress test the simplest possible model...
Alexandru Meterez, Pranav Ajit Nair, Depen Morwani et al.· arXiv.org· 3 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.