Skip to content

Toward Human-in-the-Loop Robot Failure Recovery: Bridging Communication Gaps in Human-Robot Collaboration

Sep 2026 · 0 citations · 36 references
Computer Science

TL;DR

Listener Differences in Human-Robot Interaction (LD-HRI), a game, dataset, and benchmark that evaluates speakers through human listener performance, is introduced, providing a foundation for designing more robust communication in human-robot and human-agent interaction.

Abstract

Robots can recover from failures by asking bystanders for help, but effective human-in-the-loop recovery requires communication that accounts for differences in people's knowledge. Prior inverse-semantics work generates requests using a single listener model, leaving differences in listener knowledge untested. We introduce Listener Differences in Human-Robot Interaction (LD-HRI), a game, dataset, and benchmark that evaluates speakers through human listener performance. Our evaluation examines request properties, large language model (LLM) speakers, and inverse-semantics request-selection algorithms under controlled differences in listener information. The corpus contains 446 human-written requests and 1{,}302 listener trials. We additionally evaluated 24 frozen LLM-written requests with 70 human listeners across 560 trials. Novice success is descriptively higher with model-written requests across all four tasks, yet both request sources leave substantial expert--novice gaps, including 16 percentage points for LLM requests. LD-HRI makes these gaps measurable, providing a foundation for designing more robust communication in human-robot and human-agent interaction.

View source

Similar papers

Conference Aug 2026

LLM-Assisted Planning Failure Resolution in Human-Robot Collaboration

In Human-Robot Collaboration, failures that are not explained or handled with alternative strategies can undermine user trust and interaction quality. Despite recent advances in LLM-based task planning, failures are often treated as terminal events, leading to non-transparent robot behaviors. This work investigates how...

Luca Raggioli, Emanuele Ciro Muto, Raffaella Esposito et al. · 0 citations

Delegating or Doing? Understanding User Behavior in Hybrid Human-Agent Interfaces

The findings suggest that the primary benefit of human--agent interfaces may be reducing interaction effort rather than improving speed, and that delegation reflects who the user is more than what the task demands.

Gavin Raine Dizon, Tyrone Justin Sta Maria, J. A. Deja et al. · 0 citations
Book Open access Oct 2026

Evaluating the Impact of Large Language Model Assistance on Human Attention Paradigm in Human–Robot Collaboration

Human–robot collaboration in industrial and semi-structured environments imposes significant attentional demands on the human operator, particularly during complex or time-sensitive tasks. While these collaborative robots are designed to assist the human operator, they can also introduce attentional fragmentation for h...

T. A. Chowdhury, Jose A. Trapero, F. I. Corona-Strauss et al. · 0 citations
Book Open access Aug 2026

Intent-Aware Backchanneling for Active Listening in Human-Robot Interaction

In conversational Human-Robot Interaction, robots typically remain silent during user speech and reply only after a pause, making interaction feel unnatural. In contrast, humans signal that they listen through active behavior. To overcome this, we present a system in which a social robot conveys active listening throug...

Yang Sun, Jan Leusmann, Michael A. Hedderich · 0 citations

Related blog posts

MIT News · Artificial Intelligence Nov 13, 2026

Helping robots fit into the human workplace

Founded by Anthony Jules ’92, SM ’92 and Professor Emeritus Rodney Brooks, Robust AI’s robots are designed to collaborate with humans in factories and warehouses.

Microsoft Research Blog Nov 16, 2026

TokenEconomizer: Learning from Experience to Make AI Coding Agents More Efficient

AI coding agents are increasingly help developers fix bugs and build features largely on their own, but every interaction with their models consumes tokens. In each session, an agent sends instructions, tool descriptions, and session history to its model. Most efforts to reduce this cost focus on individual tasks. Yet developers return to the same […] The post TokenEconomizer: Learning from Experience to Make AI Coding Agents More Efficient appeared first on Microsoft Research.

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.