Scientific software presents a demanding test for computer-using agents based on visual language models (VLMs): completing a research workflow requires interpreting specialized interfaces, manipulating scientific objects, and producing verifiable results. We thus introduce OSWorld-Science, a benchmark and evaluation en...
Ding-Yuan Dai, He-Li Qi, Lei Liu et al.· 0 citations
This work introduces RecreationWorld, a five-platform framework built around recreation: given a running reference, an agent must discover its behavior and build a faithful implementation with no prescribed workflow, and introduces RecreationBench, comprising 250 diverse tasks across domains and platforms.
Shuai Bai, Jia-Yong Deng, Si-Cheng Fan et al.· 1 citation
Qwen-CUA is introduced, a native computer-use agent with a 397B-A17B Qwen mixture-of-experts backbone that outperforms Qwen3.7 and remains competitive with leading proprietary systems, and scalable verifiable interaction and hybrid tool use as key directions.
Dunjie Lu, Shuai Bai, Tianyi Bai et al.· 2 citations
These results show that current agents are still far from professional-level computer use: rather than stumbling on basic GUI control or coding, they lose track of constraints, miss information that arrives mid-task, guess rather than ask the user, and skip verification, struggling most when a task hinges on hidden sta...