Listener Differences in Human-Robot Interaction (LD-HRI), a game, dataset, and benchmark that evaluates speakers through human listener performance, is introduced, providing a foundation for designing more robust communication in human-robot and human-agent interaction.
Abstract
Robots can recover from failures by asking bystanders for help, but effective human-in-the-loop recovery requires communication that accounts for differences in people's knowledge. Prior inverse-semantics work generates requests using a single listener model, leaving differences in listener knowledge untested. We introduce Listener Differences in Human-Robot Interaction (LD-HRI), a game, dataset, and benchmark that evaluates speakers through human listener performance. Our evaluation examines request properties, large language model (LLM) speakers, and inverse-semantics request-selection algorithms under controlled differences in listener information. The corpus contains 446 human-written requests and 1{,}302 listener trials. We additionally evaluated 24 frozen LLM-written requests with 70 human listeners across 560 trials. Novice success is descriptively higher with model-written requests across all four tasks, yet both request sources leave substantial expert--novice gaps, including 16 percentage points for LLM requests. LD-HRI makes these gaps measurable, providing a foundation for designing more robust communication in human-robot and human-agent interaction.
In Human-Robot Collaboration, failures that are not explained or handled with alternative strategies can undermine user trust and interaction quality. Despite recent advances in LLM-based task planning, failures are often treated as terminal events, leading to non-transparent robot behaviors. This work investigates how...
Luca Raggioli, Emanuele Ciro Muto, Raffaella Esposito et al.· IEEE International Symposium...· 0 citations
This work poses the choice as a three-action decision problem, act, consult your own sensors, or ask a human, whose optimal policy follows from measured accuracy, whose optimal policy follows from measured accuracy.
The findings suggest that the primary benefit of human--agent interfaces may be reducing interaction effort rather than improving speed, and that delegation reflects who the user is more than what the task demands.
Gavin Raine Dizon, Tyrone Justin Sta Maria, J. A. Deja et al.· International Conference on...· 0 citations
Human–robot collaboration in industrial and semi-structured environments imposes significant attentional demands on the human operator, particularly during complex or time-sensitive tasks. While these collaborative robots are designed to assist the human operator, they can also introduce attentional fragmentation for h...
T. A. Chowdhury, Jose A. Trapero, F. I. Corona-Strauss et al.· Companion Publication of the...· 0 citations
In conversational Human-Robot Interaction, robots typically remain silent during user speech and reply only after a pause, making interaction feel unnatural. In contrast, humans signal that they listen through active behavior. To overcome this, we present a system in which a social robot conveys active listening throug...
Yang Sun, Jan Leusmann, Michael A. Hedderich· Message Understanding Confer...· 0 citations
These findings motivate a state-action controller that starts from the base model action and selectively edits it using structural readouts, without requiring a complete predicted belief state as an intermediate representation.
Atahan Dokme, Larry Heck· arXiv.org· 0 citations
Related blog posts
MIT News · Artificial Intelligence· news.mit.eduNov 13, 2026
Founded by Anthony Jules ’92, SM ’92 and Professor Emeritus Rodney Brooks, Robust AI’s robots are designed to collaborate with humans in factories and warehouses.
Microsoft Research Blog· microsoft.comNov 16, 2026
AI coding agents are increasingly help developers fix bugs and build features largely on their own, but every interaction with their models consumes tokens. In each session, an agent sends instructions, tool descriptions, and session history to its model. Most efforts to reduce this cost focus on individual tasks. Yet developers return to the same […] The post TokenEconomizer: Learning from Experience to Make AI Coding Agents More Efficient appeared first on Microsoft Research.