A systematic analysis of existing evaluation frameworks and metrics for RAG-based Question Answering (QA) systems reveals urgent needs for hybrid evaluation frameworks, reasoning path traceability, combined pipeline assessment, and harmlessness evaluation, laying groundwork for the next generation of evaluation methodologies.
A novel knowledge graph-based evaluation framework is proposed introducing Semantic Structural Similarity for KGs (S3KG), a hybrid similarity measure integrating structural and semantic similarity into a continuous evaluation score, alongside a diagnostic framework for categorizing reasoning errors.
Subavarshana Arumugam, Mamta Nallaretnam, K. Wickramasinghe et al.· 0 citations
This survey provides a comprehensive review of developments in NQG, spanning traditional neural approaches to the latest paradigms driven by large language models (LLMs) and multimodal large language models (MLLMs).
A semantic correctness taxonomy is introduced that assigns open-ended answers to eight ordered classes, separating verbose-but-correct answers from those contaminated by hallucinated content and CAP (Context-Aware Precision), a reference-based metric that scores question-conditioned statements using bidirectional NLI.
Elitsa Yotkova, Violeta Kastreva, Petar Velkov et al.· 0 citations
A systematic review of 137 shortlisted research articles reveals that the performance of a cross-lingual QA system is structurally influenced by the interaction between linguistic typology, model design, and not exclusively by model scale or architecture.
