CoSy: Conversational Synthesis for Grounded Question Answering

Abstract

High-quality, large-scale conversational datasets are scarce, making it difficult to train on-device language models (OD-LLMs, ∼ 1B parameters) as effective assistants. We introduce CoSy ( Co nversational Sy nthesis), a novel framework for generating diverse, steerable, multi-turn conversations at scale. CoSy combines three key mechanisms: (1) conversational graphs that ensure natural dialogue flow, (2) turn-based prompt augmen-tations for diversity, and (3) explicit linguistic phenomena for coherence. We evaluate CoSy on conversational grounded reasoning tasks (i.e., answering questions based on contextual information), a core on-device use case. Our on-device sized models trained on CoSy-synthesized data achieve competitive performance with human-annotated baselines and outperform instruction-tuned models of up to 70B parameters in zero-shot settings.