#human-computer interaction

AgentGUI: An Interface for Observing and Steering Long-Running AI Agents

Xuan Zhao Jiwoong Sohn Qinyue Zheng Michael Moor

Jul 2026 · arXiv.org · Vol abs/2607.26300 · 0 citations · 28 references

Computer Science

TL;DR

AgentGUI is introduced, a user-friendly, locally hosted GUI for seamlessly observing and steering AI agents amid multiple concurrent, long-running sessions and features rich agent trajectory visualizations, effective manual and automated steering, and integration with and coordination between open-source and frontier agent frameworks.

Abstract

AI agents are increasingly adept at tackling complex, long-running tasks. With the rapid surge of autonomous capabilities, human oversight is systematically lagging behind due to limited human-centered interfacing. Aiming to address this, we introduce AgentGUI, a user-friendly, locally hosted GUI for seamlessly observing and steering AI agents amid multiple concurrent, long-running sessions. AgentGUI features 1) rich agent trajectory visualizations, 2) effective manual and automated steering, and 3) integration with and coordination between open-source and frontier agent frameworks. A controlled user study demonstrates statistically significant reduction in the time it takes to identify key elements from agent traces (38% faster, p = 0.023). In a preliminary experiment, AgentGUI's automated drift prevention feature raises the task completion rate of small local agents by as high as 34pp across a 0.8B--9B model ladder (N=50 runs per model). AgentGUI is publicly available through its project website (https://agent-gui-project.github.io) and open-source repository (https://github.com/eth-medical-ai-lab/agent-gui), along with a demo video (https://youtube.com/watch?v=GSDyxN1gTF0).

DOI arXiv Semantic Scholar

View source

Similar papers

#artificial intelligence Preprint Sep 2026

Toward Self-Adaptive Physical AI: Can LLM Agents Manage Long-Horizon Physical Tasks?

The results show that zero-shot LLM agents can achieve comparable management outcomes to RL agents under the same weather pattern and adapt more effectively than RL when evaluated under a shifted environment, highlighting a promising path toward self-adaptive physical AI agents.

V. Kaushik, Ya-Yun Tan, Xiao-Fan Yu · 1 citation

#natural language process... Preprint Aug 2026

Agents in the Large: Perception-Centered Architecture for Persistent Agents

Pera describes a persistent agent organized around perception and control components that continually perceive service-relevant signals from episodic task executions, internal context, and changes in the surrounding environment, and use these signals to construct lifecycle tasks.

Shi-Han Dou, Haoxiang Jia, Shichun Liu et al. · 1 citation

#natural language process... Preprint Oct 2026

Scaling Agentic Data for Long-Horizon Terminal Intelligence

Terminal agents are moving beyond local operations toward completing real-world professional tasks. Short trajectories can demonstrate commands and tool calls, but they do not fully show how agents accumulate evidence through exploration, use it to inform judgments, continually adjust strategies based on feedback, and...

Shun Zou, Zi-Yu Ma, Yi Zou et al. · 0 citations

Preprint Sep 2026

AgentWorld: Benchmarking Long-Horizon Collaboration of Multi-agent LLMs

To quantify collaboration effectiveness in addition to conventional binary task success, Causal Collaboration Effectiveness (CCE) is proposed, a graph-based metric that traces causal dependencies between agent actions and measures what fraction of a team's effort actually contributed to the outcome.

Raphael Shu, Yu-Sen Zhang, Y. Cho et al. · 2 citations

Review Open access Aug 2026

From Language Models to Agentic AI: A Survey of Autonomous, Action-Enabled, and Collaborative LLM Agents

A unified, taxonomy-driven, and deployment-oriented survey of agentic AI systems, synthesizing recent advances through a modular reference architecture and a four-dimensional taxonomy that characterizes agents along the axes of autonomy, tool use, collaboration, and safety–governance is presented.

Sparsh Bajoria, Shreyanshu Ranjan, Adhitya M et al. · Cognitive Computation · 1 citation

#artificial intelligence Preprint Oct 2026

Cooperating with Future Collaborators: Multi-Agent RL under Staggered Participation

In cooperative Multi-Agent Reinforcement Learning (MARL), agents are often trained under concurrent participation, while in many tasks some agents act earlier and leave task-relevant information that becomes useful to agents participating later. We study this setting as staggered participation (SP), which introduces a...

Jiang-Lin Qiao, Si-Yi Hu, Thien Hoang Nguyen et al. · 0 citations