Jun 2026· ACM Trans. Inf. Syst.· Vol 44, pp. 1 - 29· 0 citations· 83 references
Computer Science
TL;DR
A large language model (LLM)-based three-stage framework that relies on an existing community question answering dataset is proposed, showing that the three-stage generation approach consistently outperforms the baseline particularly in recall, while also achieving competitive results across other metrics.
Abstract
Conversational search addresses users’ information needs through multi-turn and context-aware interactions. Given that user queries are often ambiguous, the use of clarifying questions can effectively reduce uncertainty and enable a mixed-initiative conversational system. However, current datasets for clarifying questions remain limited in the following three aspects: (1) underrepresented multi-turn conversational data, (2) limited diversity, and (3) heavily reliance on crowdsourcing, thereby suffering from limitations such as high annotation cost. To address these issues, we propose a large language model (LLM)-based three-stage framework that relies on an existing community question answering dataset. It encompasses: (1) extracting essential information from the initial user query with the relevant contextual information, (2) generating clarifying questions paired with corresponding answers, and (3) refining conversations to ensure coherence and a natural conversational flow. We assess our multi-stage method against a baseline that directly prompts LLMs to generate conversations in a single-step process, evaluating on an answer retrieval task using recall, precision, normalized discounted cumulative gain and mean average precision. Results show that our three-stage generation approach consistently outperforms the baseline particularly in recall, while also achieving competitive results across other metrics. Human and automatic evaluations further indicate the high quality of generated conversations and fine-tuning on them improves retrieval performance, highlighting the pipeline’s potential.
Intelligent Virtual Agents need to be able to participate in extended dialogue interactions while maintaining a conversational style. We discover the elements of conversational style in open-domain dialogues by analyzing the features that distinguish conversational system responses from responses that are merely topically relevant. We first collect 7,751 dialogue contexts from live human conversations with a multi-generator Alexa Prize SocialBot that was deployed across four competition years. We also collect 35,623 candidate responses for the contexts. We annotate the responses with a four-level ABCD quality scheme that isolates conversational naturalness from topical relevance. We then extract twenty-five linguistic features that capture conversational properties of responses and contrast (A) responses that have a conversational style, from (B) responses that are topically relevant, but less conversational. We find that conversational responses are marked by other-directed engagement: question-asking, user engagement phrases, acknowledgment openings, and second-person reference, while responses that are merely relevant and topical are marked by self-oriented information delivery: opinion markers, formulaic openings, and hedges and emphasizers deployed in service of the system’s own assertions. We thus find that conversational style in this setting is best understood as a pragmatic orientation toward the user rather than toward the system’s own content, and that the system must be mixed-initiative to manifest a conversational style. We discuss what these findings imply for the design of Intelligent Virtual Agents.
Vrindavan Harrison, M. Walker· Proceedings of the 26th ACM...· 0 citations
This study directly test whether that omitted within-conversation context changes answers in a conversation and concerns preceding turns in the same conversation and does not test persistent memory across separate conversations.
We present the insights gathered at the 1st Workshop on Conversational Search for Complex Information Needs (CoSCIN'26) held on 2 April 2026, in conjunction with the Forty-eighth European Conference on Information Retrieval (ECIR). The workshop brought together keynote presenters and industry representatives for lively discussions on recent developments in conversational search and its future directions, complemented by Q&A sessions for paper presentations. The rapid developments in conversational search enabled interesting discussions that went beyond generative dialogue and factoid question-answering tasks, towards addressing complex information needs and improving the user experience by providing personalized, adaptive answers to users based on style. Moreover, the work that was presented supported exploratory, multi-step information needs and addressed challenges such as longform answer generation, personalization, agent orchestration, and societal considerations, which remain largely open areas of research. The participants shared their views from diverse backgrounds in information retrieval, natural language processing, and human-computer interaction, exchanged results and prospective ideas around conversational AI, and discussed its impact on users and the need for adaptivity. Through these discussions, the workshop fostered collaboration and connected researchers specializing in diverse facets of the topic, ultimately advancing the field toward creating more user-centric, intentionally designed, responsible conversational systems. Date: 2 April 2026. Website: https://convsearch-complex-info-needs.github.io/.
Roxana Petcu, M. Yazan, Mohanna Hoveyda et al.· ACM SIGIR Forum· 0 citations
This work proposes a new approach that quantifies the effectiveness of each interaction by the reduction in the assistant's uncertainty, measured via entropy over recommendations, to fine-tune the LLM, enabling strategic interaction generation.
Cedar Site Bai, Zhenyu Liao, Duan Li et al.· 0 citations
Adaptive Personalised Reranking (APR) is proposed, a framework that routes queries based on intent that uses efficient similarity-based reranking for simple queries and dynamically generates tailored instructions to guide an instruction-following reranker for constraint-heavy contexts.
