Skip to content
Conference

RAG-Based Emotion-Aware Conversational Agent with Multilevel Sentiment and Pragmatic Contextual Analysis

Aug 2026 · International Conference Computational Vision and Bio Inspired Computing · pp. 640-645 · 0 citations · 26 references

Abstract

Retrieval-Augmented Generation (RAG) has significantly reduced hallucination rates in Large Language (models, or LLMs). Standard RAG workflows (models, or LLMs) remain unfeeling to user affect, often giving responses which are true but emotionally inappropriate. This paper introduces an Emotion-Aware Conversational Agent that embeds Multilevel Sentiment and Pragmatic Contextual Analysis into the RAG orchestration layer. We design a highly modular pipeline built on a Multi-Task Learning (MTL) Transformer backbone that jointly classifies user emotion and pragmatic intent in real time. To reduce mode collapse arising from skewed emotional class distributions, we employ an inverse-frequency weighted Cross-Entropy loss when fine-tuning the classification head. The inferred psycholinguistic state is tracked through a temporal Dialogue State Tracker (DST) and combined with vector-based domain retrieval, enabling dynamic adjustment of the generative prompt. To curb run-on script hallucinations, the orchestration layer natively adopts ChatML templating. In local evaluation using the RAGAS framework with a 4-bit double-quantized Qwen2.5-1.5B generator, our architecture achieves a 0.84 Macro F1-Score on minority affect classes and preserves strong factual faithfulness (0.87) while markedly improving Answer Relevancy (0.91). These results demonstrate that deeply contextual, emotion-aware AI systems can be efficiently deployed on edge devices without depending on proprietary cloud APIs.

View source

Similar papers

Open access 2026

Explainable Context-Aware Emotion Recognition in Conversations Through Attribution Analysis, Layer-Wise Probing and Representation Geometry

Emotion recognition in conversations is a fundamental capability for advanced human–machine interaction. Its main aim is to enable dialogue systems to adapt their content, tone and pacing to users’ affective state and improve engagement, satisfaction and trust. In this study, we implement and evaluate emotion classific...

Isidoros Perikos, Theodoros Kontopoulos · 0 citations
Open access Sep 2026

A Hybrid Framework for Real-Time Sentiment Analysis and Adaptive Conversational Outcome Prediction in Domain-Specific Dialogue Systems

The lack of non-verbal signals in text-based communication forces dialogue systems to rely entirely on dynamic textual sentiment changes to infer user intent and emotional shifts. However, substantial class imbalance and the static, post-hoc nature of existing text analysis models prevent dialogue systems from recogniz...

Mejdl S. Safran · 0 citations
Open access Sep 2026

Beyond Labels: Training Cognitive Empathy with Context-Rich Synthetic Dialogues

RelationalDialogues is introduced, a novel, fully synthetic dataset of 12,849 multi-turn dialogues designed to explicitly train perspective-taking, demonstrating that training on highly contextualized, metadata-driven synthetic data is an effective method for advancing LLMs from displaying superficial sympathy to engag...

Neema Owji, Cameron Buckner · 0 citations
Open access 2026

CEM MoE: A Lightweight Sparse Mixture of Experts Framework for Empathetic Dialogue Generation

Empathetic dialogue generation requires models to understand user emotions and produce fluent, relevant, and emotionally appropriate responses. Existing lightweight models offer favorable deployment efficiency, but shallow emotion prediction layers limit their ability to represent fine-grained and dynamically changing...

Zhi-Nan Gou, He Liu, Yufan Wang · 0 citations
#artificial intelligence Preprint Sep 2026

Exposing Weaknesses in Emotion Recognition in Conversations

An LLM-as-Judge framework is introduced that evaluates each emotion independently according to its plausibility in the conversational context rather than enforcing a single-label decision, suggesting that standard single-label evaluation is therefore insufficient.

Amir Ben Khalifa, Fanny Bezancon, B. Abdulrazak et al. · 0 citations
#artificial intelligence Preprint Sep 2026

Beyond Text: LLM-Based Dimensional Emotion Evaluation in Multimodal Dialogue

Emotion recognition in conversation has been widely studied, but applying Large Language Models (LLMs) to continuous dimensional emotion evaluation in multimodal dialogue remains largely unexplored. We propose an LLM-based framework that performs discrete emotion recognition and Valence-Arousal-Dominance (VAD) dimensio...

Yu-Tong Hu, Jin-Ho D. Choi · 0 citations

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.