AcCoRD: Evaluating User-Agent Collaboration Under Realistic User Preference Dynamics
This work introduces AcCoRD, a user-agent collaboration benchmark requiring agents to handle diverse user preference dynamics in two domains: online shopping and travel planning, and evaluates five frontier LLMs under two prompting strategies: vanilla ReAct, and an uncertainty-guided variant that prompts models to identify and resolve ambiguity about user preferences.
Tejas Srinivasan, Shikib Mehri, Nandita Shankar Naik et al.
· 0 citations