SADA-RAG: Staged Adaptation and Deterministic Structured Evidence Augmentation for GEO-OSINT Question Answering
Abstract
Question answering over public, historical open sources requires reliable retrieval behavior and access to exact structured records. We present SADA-RAG, a retrieval-augmented question answering (QA) framework for GEO-OSINT question answering over public historical sources that combines staged agent adaptation with deterministic structured evidence augmentation. Qwen2.5-7B-Instruct is first adapted to the retrieval protocol and subsequently adapted to the target domain using low-rank adaptation (LoRA), separating retrieval-interaction learning from domain alignment. At inference time, a deterministic router retains dense retrieval for narrative evidence and activates the structured branch only when a question depends on a target source. Source-specific exact matching and conservative guards determine whether matched evidence is injected. We evaluate the framework on a locked geopolitical open-source intelligence (GEO-OSINT) benchmark built from public historical sources. Relative to the unadapted Qwen2.5-7B-Instruct agent, the full pipeline improves raw exact match (Raw EM) by 15.29 points, with model-side adaptation and structured evidence augmentation contributing comparable gains. Over the adapted agent without structured evidence, Raw EM increases from 0.3654 to 0.4419 while dense-search count remains nearly unchanged, with only a modest increase in end-to-end evaluation time. On non-target questions, the main-line variants produce identical predictions, with no observed answer-level degradation. Controlled ablations and retrieval-backend controls further isolate the contributions of staged adaptation and selective structured-evidence access from the effects of retrieval breadth and backend choice. Together, the results support precision-oriented structured evidence augmentation for auditable question answering over public historical sources.