Evaluation has been central to advances in AI, and it is equally critical to autonomous discovery. Yet scaling autonomous discovery to real-world engineering challenges faces a fundamental bottleneck: the cost of evaluating progress. In real settings, a single true evaluation, such as a full benchmark or serving trace,...
Shubham Agarwal, Akshat Gupta, M. Heddes et al.· 0 citations
Reward maximization alignment methods for discrete diffusion models have primarily focused on steering the reverse process, either by influencing token logits or by selecting favorable sequences at intermediate steps. These approaches largely treat inference as a unidirectional process, lacking mechanisms for revisitin...
Shai Dickman, M. Cemri, Landon Butler et al.· 0 citations
Software development follows an implementation-verification loop in which developers or agents iteratively revise an implementation until an evaluator, such as a test suite, accepts it. The evaluator checks the implementation against a set of requirements under a model of the deployment environment. Yet even a formal p...
Alexander Krentsel, Shubham Agarwal, M. Cemri et al.· 0 citations
It is argued that an agent system should maintain an explicit representation of how it fails, induced from its own behavior and reusable wherever failure feedback is needed, and AdaMAST builds this representation by converting a target system's traces into a compact, evidence-grounded failure taxonomy.
M. Cemri, Andrei Cojocaru, Melissa Z. Pan et al.· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.