Skip to content
Book Open access

Emerging Risks of Conversational AI

Jul 2026 · International Conference on Conversational User Interfaces · 0 citations · 73 references
Computer Science

TL;DR

This workshop aims to gather a multidisciplinary community to reflect on pressing challenges and uncertainties and explore risk analysis and mitigation strategies in CUI design and deployment and build an international network of scholars and practitioners to promote responsible and human-centred conversational AI.

Abstract

Conversational User Interfaces (CUIs) are rapidly advancing, moving from single-task assistants to powerful and engaging artificial agents. As CUIs become embedded in daily life, the implications for users interacting with such systems demand closer scrutiny. While some issues are immediately identifiable (e.g. privacy risks, misinformation, AI hallucinations), others may only become visible after longer periods of use, including over-reliance, erosion of human agency, or the normalization of biased or exclusionary language. This workshop aims to gather a multidisciplinary community to reflect on pressing challenges and uncertainties and explore risk analysis and mitigation strategies in CUI design and deployment. Through presentations, discussions, and collaborative design activities, participants will examine immediate and longitudinal risks and share empirical insights. The activities will support developing interdisciplinary frameworks to understand and handle unintended consequences of CUIs. The workshop seeks to build an international network of scholars and practitioners to promote responsible and human-centred conversational AI.

Read PDF

Similar papers

Jul 2026

Informal Learning Emerges in Everyday Human-LLM Interaction

As LLMs become increasingly capable of completing tasks for users, a central concern is that everyday AI use may become primarily cognitive offloading, eroding the opportunities through which people develop their own capabilities. We analyse large-scale human-LLM conversations to ask whether informal learning behaviors also emerge in this setting: whether users engage in exchanges in ways that preserve opportunities to learn. Across 128,569 naturalistic conversations, we translated learning-science constructs into turn-level behavioural signatures. Cognitive engagement, users'cognitive effort as reflected in the exchange, appeared in 31.9% of 491,685 user turns, whereas constructive engagement, the deepest observable form of learning-oriented engagement, appeared in 4.9%, showing that deeper sense-making was recurrent but selective. Our study further identifies factors associated with these forms of engagement. Scaffolded assistant support consistently marked richer constructive participation, with associations varying by user framing, task ecology, support form, timing and prior user state. Together, these findings show that everyday human-LLM interaction is not only answer delivery or cognitive offloading; it also contains measurable, selective and conditionally organized behavioural signatures of informal learning. They shift AI evaluation from answer-delivery efficiency toward the preservation of cognitive opportunities for users to reason, test ideas and construct understanding in the course of everyday problem-solving.

Zixin Chen, Haotian Li, Ziang Xiao et al. · 1 citation
#human-computer interacti... Preprint Jul 2026

User-Side Contextual Phenomena in Long-Term Human-AI Interaction

Current assessments of conversational AI focus mainly on model outputs, including hallucinations and factual errors. These measures matter, but this paper examines risks that may form on the user side during long and repeated interaction. The study follows one user across nearly four thousand conversations with the same system over twenty months. The user gradually interpreted the system as having memory, care, judgment, and authority, and reorganized part of their thinking around it. A single response may show no clear problem, while long and frequent interaction can still create another layer of risk. This paper calls that layer User-Side Contextual Phenomena (USCP) and examines records from August 2024 to April 2026. The study uses an exploratory single-case longitudinal qualitative design with autoethnographic positioning. A hybrid deductive-reflexive thematic approach organizes the material into three main modes: contextual projection, contextual attachment, and contextual authority transfer. The paper does not estimate prevalence, make diagnoses, or validate an instrument. It offers a non-clinical vocabulary and four evidence roles: inclusion, gray-zone, negative, and protective gray-zone. Its central claim is that an acceptable response on its own does not establish safety across a series of conversations. User-side risk can still form during long-term interaction.

Zon Rzvn · 0 citations
#generative ai Preprint Aug 2026

AffAdapt: AFFect-driven ADAPTive AI Personas for Seamless Conversations

AffAdapt is presented, a seamless interaction design framework for AI-personas, which coordinates streaming speech recognition, proactive turn-management, persona-grounded response generation, a persistent emotional state, and synchronized embodied output into a single interaction loop.

Nishanth Chidambaram, Kaustubh Paliwal, Kayla Hom et al. · 0 citations
Book Open access Jul 2026

ADAPTIC: Adapting Dialog and Pragmatic Traits in Context

This workshop proposes a paradigm shift toward designing context-specific CUIs that actively support communicative success through pragmatic cues, and brings together perspectives from HCI and social sciences to explore how users appropriate conversational AI across domains.

Laura Spillner, Johanna Rockstroh, Paul Goerke et al. · 0 citations
Jul 2026

Psychological Influences of Conversational AI: Research and Design Directions for Reducing Harm and Promoting Well-Being

This work proposes a set of aspirational directions for guiding the behavior of general-purpose AI systems in ways that may reduce potential psychological harms and support user well-being, and identifies open questions and areas requiring deeper study.

Ji-Na Suh, Mihaela Vorvoreanu, Forough Poursabzi-Sangdeh et al. · 0 citations
Book Open access Jul 2026

KNIT-C: Exploring Conversational AI Facilitation of Group Convergence Through Computational Boundary Objects

Conversational AI is widely explored for supporting brainstorming and creative collaboration, yet its potential to guide groups through convergence, where differing perspectives must be reconciled into a shared direction, remains largely unexamined. We introduce KNIT-C, a conversational system that operationalises computational boundary objects within synchronous multi-user dialogue, eliciting individual viewpoints, synthesising group perspectives, and guiding discussion toward negotiated outcomes. Across four public workshop sessions (14 non-expert participants), we find that conversational engagement deepened individual reflection but created pacing and coordination challenges in group interaction. AI-generated summaries anchored discussion, though their similarity demanded careful comparison rather than simplifying choices. Participants frequently reported feeling heard without fully endorsing outcomes, pointing to productive disagreement as a form of convergence that preserves visible tension rather than resolving it. We discuss implications for adaptive pacing, context-sensitive stance-shifting, and evaluation frameworks that assess whether differences were genuinely engaged rather than suppressed.

Echo Wan, C. Mougenot, Malak Sadek · 0 citations

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.