A Shared Task on Agentic AI in University Administration
Abstract
Agentic AI systems combine large language models, information retrieval, planning, and tool use to act on behalf of users in complex digital environments. Evaluating such systems under laboratory conditions remains challenging because real-world environments are hard to control: executing AI agents in them is neither safe nor reproducible. We show that university administration provides a particularly suitable setting for Cranfield-style evaluations of AI agents in a laboratory. By reconstructing the digital environment available to university staff, we enable the systematic evaluation of interactive AI agents within human administrative workflows. TIRA's sandboxed execution environment allows submitted agents to operate safely on institutional data. Focusing on selected workflows, such as travel reimbursement, we organize the first shared task on agentic AI in university administration and contribute a reusable benchmark with expert-judged reference solutions. The shared task comprises two subtasks: in agentic retrieval, an agent only finds the documents relevant to a task instance; in agentic task solving, the agent also performs the task and produces the solution.