LibraryDesignBench is introduced, a two-phase benchmark in which an agent implements a full-featured library from a specification that defines required capabilities and potential use cases without prescribing the design.
Gabriel Orlanski, Alex L. Zhang, Avi Trost et al.· 0 citations
An LLM agent framework, JAZ, is built to explore the extent to which a minimal harness that is little more than the agent loop itself can accomplish tasks these specialized systems are built for.
Zhe-Ning Li, Joshua Liu, M. Vukelic et al.· 0 citations
Low-friction, expressive membrane prevents harness failures from becoming model failures and pushes measurement toward the model's true maximal underlying capability, on Factorio, where refinement allows for continuous technology progression and dedicated subagents enable parallelized work.
Seth Karten, Alex L. Zhang, Kevin Thomas et al.· 16 citations· ⚡2
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.