Persistent Goal-Tracking and Instruction-Driven Reasoning in Sequential Conversation QA
TL;DR
GSC-QA (Goal-based Sequential Conversation QA), a framework that integrates three complementary components into a unified enterprise dialogue architecture that combines retrieval, instruction enforcement, and goal persistence in a single coordinated loop built on LangGraph, is introduced.
Abstract
Traditional conversational Question Answering (QA) systems focus on isolated information retrieval and lack the ability to maintain goal persistence across multi-turn interactions. Current Retrieval-Augmented Generation (RAG) approaches operate passively, retrieving information turn-by-turn without tracking whether the overall task objective has been achieved, and are typically evaluated using linguistic similarity metrics rather than functional task outcomes. We introduce GSC-QA (Goal-based Sequential Conversation QA), a framework that integrates three complementary components into a unified enterprise dialogue architecture: 1) persistent Goal Tracking Module (GTM) that maintains task state across turns in external databases (Redis + PostgreSQL), decoupled from the LLM context window; 2) Expert Instruction Layer that injects domain-curated procedural constraints into every response generation step; and 3) proactive slot detection that identifies and elicits missing required information before proceeding. Unlike prior task-oriented systems that assume static ontologies or session-scoped memory, GSC-QA combines retrieval, instruction enforcement, and goal persistence in a single coordinated loop built on LangGraph. Evaluated on 1,516 annotated dialogue records in Vietnamese spanning ten enterprise domains (principally beauty services, professional services, fashion & retail, spa & wellness, and language education, with smaller deployments in marketing, healthcare, banking, and lighting & electrical equipment), GSC-QA achieves a Task Success Rate (TSR) of 87.1% and a Goal Consistency Score (GCS) of 0.78 on the combined 1,516-record evaluation set, and an Error Recovery Rate (ERR) of 79.5% on 205 organic pivot transitions, substantially outperforming standard RAG (58.7% TSR, GCS 0.62). All baselines including Instruct-Only are evaluated on the same dataset for fair comparison. Cross-domain evaluation across 980 records from 14 enterprise deployments shows the mechanism transfers with little loss: on a curated 299-record subset scored under the same judge and rubric as the primary set, task success reaches 62.2% against 63.2% in-domain. Against four external baselines, including an agentic ReAct system, a memory-augmented long-context model, a SimpleTOD-style tracker and a graph-structured stateful agent, GSC-QA leads on task success, faithfulness and answer relevancy alike, and its lead in task success holds under four independent judge families. An LLM-as-judge metric (gpt-5.4) achieves 94% agreement with human annotations. Ablation studies confirm that goal persistence and expert instructions are complementary; removing either component causes significant performance degradation.