Large language model (LLM) agents increasingly complete tasks through live graphical user interfaces (GUIs). Because the model cannot directly operate the GUI, these systems often rely on tools to capture screenshots or UI trees, translate model outputs into GUI actions, and report whether the interface changed. We stu...
Yu-Chen Shao, Wen-Jing Liu, Yu-Heng Huang et al.· 0 citations
Code embedding models have become a key building block for many software engineering applications, yet their internal representations remain poorly understood. Existing evaluations tend to focus on downstream task performance, offering limited insight into whether a single embedding can support both functional abstract...
Extending Vision-Language-Action models (VLAs) to humanoid robots requires coordinated whole-body control and diverse demonstrations that are costly to collect in real-world scenarios. Although egocentric human demonstrations provide a complementary source of task experience, differences in visual embodiment and missin...
Yang Liu, Jia-Cheng Liu, Heng-Tao Li et al.· 1 citation
Datura, an automated red teaming testing framework that exposes safety vulnerabilities through chained tool manipulation, is proposed and evaluated across five LLMs and 740 safety-critical tasks under five defense settings, including real-world safety mechanisms.
Yu-Chen Shao, Zi-Qun Bao, Yu-Heng Huang et al.· Proceedings of the ACM on So...· 0 citations
This work introduces Code-MUE, a purely black-box framework that measures uncertainty through execution-based Semantic Interaction Graphs, and grounds uncertainty in observable runtime behavior, calculating the Von Neumann entropy of the solution space to quantify global semantic diversity.
Xiao-Ning Ren, Yin-Xing Xue, Lei Ma et al.· arXiv.org· 0 citations
NARU, a benchmark designed to evaluate Narrative evolution and Reasoning on cultural Understanding in Japanese long-form video, is introduced, a hierarchical memory-based annotation pipeline that transforms raw video into structured event, narrative, and cultural annotations, then generates questions via task-oriented...
Yu-Heng Huang, Jian-Lang Chen, Jia-Yang Song et al.· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.