Users are increasingly delegating tasks to AI agents, allowing agents to perform long-running, multi-step workflows on their behalf. However, agents may perform these tasks in ways that depart from users'expectations for how they should be carried out, leading to what we call agentic delegation failures. We leverage a...
Shravika Mittal, Sunnie S. Y. Kim, Scott Counts et al.· 0 citations
Users increasingly rely on AI systems for extended, multi-turn tasks, yet evidence indicates that LLM-based system performance degrades over prolonged interaction. To evaluate and improve AI assistants in such settings, we need user simulators that faithfully follow complex user intents while exhibiting realistic conve...
Mustafa Omer Gul, Philippe Laban, Tanya Goyal et al.· 0 citations
User simulators enable interactive training and evaluation of AI assistants at scale. Yet current simulators struggle to reproduce the diverse behaviors of real users, leading to assistants that generalize poorly to underrepresented users. We identify user representations as an underexplored bottleneck in user simulati...
Shuhaib Mehri, Philippe Laban, Michel Galley et al.· 0 citations
This work introduces a framework that transforms static, single-turn tasks into dynamic multi-turn conversations in which the user's intent evolves across turns, while preserving each task's original evaluation protocol, enabling existing benchmarks to be reused as controlled testbeds without new annotation.
Jihoon Tack, Philippe Laban, Jennifer Neville· arXiv.org· 4 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.
