A New Metadata-Aware Retrieval-Augmented Generation (RAG) Architecture for Trustworthy Legal Question Answering

Abstract

Large Language Models (LLMs) offer strong capabilities for Natural Language Processing, yet their inherent uncertainty often produces hallucinations, confident but incorrect statements, which is critical in domains requiring precise knowledge representation. Retrieval-Augmented Generation (RAG) reduces this risk through information retrieval, but standard pipelines still suffer from fragmented context and weak alignment between queries and legal provisions, limiting trustworthy knowledge extraction. This study proposes a Metadata-Aware RAG architecture to improve grounding in large legal corpora. It integrates: 1) Sub-chunking with Legal Metadata Inheritance, which transforms unstructured legal PDFs into granular, metadata-rich fragments; and 2) an Adaptive Filter Creator, a pipeline that extracts structured constraints and compiles optimized hybrid retrieval queries. These components enhance semantic alignment, reduce uncertainty-driven hallucinations, and strengthen neural information retrieval. Using a curated Peruvian labor law corpus and 150 manually validated question–answer pairs, the system was evaluated across three LLMs (Llama-3.1-8B, GPT-OSS-20B, Gemma-3-27B). The proposed architecture achieves double-digit improvements over a Naive RAG baseline across all four RAGAS metrics—Context Precision, Context Recall, Factual Correctness, and Faithfulness—with gains ranging from 13.10% to 28.15%; notably, Faithfulness surpasses 0.90 for Gemma-3-27B. Statistical analysis confirms significance (t(11) = 15.49, p = 4.06 × 10−9) with an extremely large effect size (Cohen’s d = 4.47). Regression results show minimal influence of model size (slope < 0.005), indicating that retrieval design has a stronger influence than parameter count in the evaluated setting.