VeriSoftBench is introduced, a benchmark of 500 Lean 4 proof obligations drawn from open-source formal-methods developments and packaged to preserve realistic repository context and cross-file dependencies and results show that success is strongly correlated with transitive repository dependence.
Randomized YaRN is proposed, a training method that improves length generalization by combining YaRN-based positional extrapolation with randomized positional encoding and a length curriculum, and suggests that progressively exposing models to OOD positional distributions provides an effective recipe for generalizable...
It is found that simply training models on CoT data of atomic tasks leads to limited generalization, but minimally modifying CoT formats of constituent atomic tasks to be composable can lead to improvements.
Fangcong Yin, Zeyu Liu, Liu Leqi et al.· arXiv.org· 1 citation
This work introduces a new formulation of G-V consistency that involves a principled correction for utterance frequency, and shows that under a natural model of rational agents answering questions with multiple answers, consistency of the validator with a frequency-corrected generator score emerges naturally.
Juan Diego Rodriguez, Jocelyn Zhang, Katrin Erk et al.· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.
