GameBoyWorlds is introduced, a testbed for agentic self-improvement in video games and it is shown that contemporary approaches to self-improvement are lacking, with world modelling and autonomous skill discovery failing, and a novel strategy that uses curiosity-based exploration to write guides achieving only partial...
Dhananjay Ashok, Adam Shen, A. Feng et al.· 0 citations
As agents are deployed with increased autonomy, even extremely rare events along their stochastic output trajectories can occur and prove catastrophic. Safe deployment therefore does not depend on whether these events can occur, but on how often they might. We study the problem of estimating the probability of rare eve...
Han-Ming Yang, Daksh Mittal, Jing Dong et al.· 0 citations
This work provides one of the first evaluations of agents on end-to-end spreadsheet tasks, focusing on economically critical financial workflows such as modeling and scenario analysis, and develops an evaluation taxonomy comprising three dimensions: Accuracy, Formula, and Format, each comprising fine-grained criteria t...