PACE: Persona Adaptation through Conversational Elicitation in Human-Robot Interaction

AffAdapt: AFFect-driven ADAPTive AI Personas for Seamless Conversations

AffAdapt is presented, a seamless interaction design framework for AI-personas, which coordinates streaming speech recognition, proactive turn-management, persona-grounded response generation, a persistent emotional state, and synchronized embodied output into a single interaction loop.

Toward User-Mediated Self-Repair in Ubiquitous Robots Through Goal-Oriented Agentic AI

Intent-Aware Backchanneling for Active Listening in Human-Robot Interaction

In conversational Human-Robot Interaction, robots typically remain silent during user speech and reply only after a pause, making interaction feel unnatural. In contrast, humans signal that they listen through active behavior. To overcome this, we present a system in which a social robot conveys active listening through non-verbal backchannels grounded in interactional intents. The system combines two ideas: (i) a dual-stage framework separating the user’s communicative intent (Speaker Intent) from the robot’s interactional stance (Listener Intent), mapping the latter to non-verbal reactions; and (ii) a parallel pipeline whose chunk-level branch generates non-verbal feedback during speech while a turn-level branch produces the verbal reply at turn end. We deployed this system on a robot and conducted a usability study (N = 6) in a hotel-negotiation task. We found that participants considered the system usable and could interpret gestures. We contribute a ready-to-deploy intent-aware system to enable active listening for robots.

The World According to a Social Robot - Augmenting Human-Robot Dialogue With Vision Language Models

Vision Language Models (VLMs) enable robots to visually perceive their environment as well as the actions and characteristics of their conversation partner or humans in collaboration. Especially for social robots deployed in everyday settings and for uncomplicated, natural use, it is essential that the robot has an understanding of situations that is appropriate to human customs. This paper presents initial experiences with the application of a Mistral AI language model with a Pepper robot for Human-Robot Interaction (HRI) in dialogue, as well as an investigation of the effects of additional visual information on response time in different models. The results show that incorporating visual information adds context to the dialogue with only a moderate increase in response time, enabling both the robot and the human to take into account unspoken elements of the situation. Furthermore, using an LLM hosted in Europe offers a solution that complies with European data protection regulations and can therefore facilitate real-life applications more easily.

Affect-aware conversational adaptation in mixed reality procedural tasks

Conversational agents based on Large Language Models (LLMs) are increasingly explored in eXtended Reality (XR) applications that require real-time guidance during task execution. In task-oriented XR environments, conversational support must remain responsive and integrated with the ongoing activity, minimizing interruptions in user interaction. Although affect-aware adaptation has been proposed as a possible strategy to improve interaction quality, its impact in immersive task-oriented scenarios remains unclear. This paper presents a mixed-reality conversational system that integrates voice interaction, LLM-driven dialogue, and prosody-based affect-aware adaptation. The system adopts a modular client–server architecture with parallel semantic and affective processing pipelines, enabling emotion-related cues inferred from vocal prosody to be incorporated without interrupting conversational flow. The system was evaluated through a between-subjects study involving 40 participants performing a guided chemical procedure within a mixed-reality laboratory scenario. Participants interacted with either an affect-adaptive or a non-adaptive version of the conversational agent. The evaluation combined standardized questionnaires, behavioral metrics extracted from interaction logs, and qualitative feedback. Results showed that the affect-adaptive condition produced greater perceived usability scores compared to the non-adaptive condition. Participants interacting with the adaptive agent also required longer task completion times and engaged in a higher number of conversational turns. No significant differences emerged for overall workload between the two conditions. Qualitative findings further suggested that affect-aware adaptation made the interaction feel more natural and supportive.

Pluralistic Human-Robot Interaction: Designing for Robot Interaction with Diverse Communities

Social robots are being developed for homes, schools, and other environments where they will interact with diverse users. While Human-Robot Interaction (HRI) research often emphasizes natural communication, engagement, personalization, and task success, these goals do not fully address the social complexity of real-world deployment. This paper proposes \emph{Pluralistic HRI}, a framework for designing social robots that treat human diversity as a foundational design concern. The framework brings together pluralism, civic dialogue, perspective-taking, empathy, intercultural competence, cultural humility, and moral imagination to guide inclusive, adaptive, and ethically grounded interaction. We outline how pluralistic HRI can inform design, evaluation, and deployment in diverse human communities.