Structure-Aware Hybrid Representation for Retrieval-Augmented Generation Over OCR-Derived Documents

Abstract

Retrieval-augmented generation (RAG) improves the performance of large language models (LLMs) by integrating external evidence for response generation. However, many RAG studies assume clean, well-structured text, whereas real-world knowledge is often sourced from optical character recognition (OCR), where layout information, field-value associations, and entity-level relationships may be lost. This study investigates the effect of document representation on RAG when applied to OCR-derived financial forms and proposes a KV-ER Hybrid representation that combines key-value and entity-relation structures. The method combines explicit field-value alignment from key-value evidence with subject-relation-value semantics from entity-relation evidence. When applied to the full question-answering (QA) evaluation set at $k=25$ , the KV-ER Hybrid achieved the best answer-generation performance, improving F1 from 0.354 to 0.476 and exact match from 0.317 to 0.429 compared to the strongest single-component baseline. The hybrid representation also achieved the best retrieval performance, with a Hit@25 of 0.866 and a mean reciprocal rank of 0.241. As additional diagnostics, the evidence-unit density and computational cost were examined. For secondary analyses using a balanced QA subset, the component complementarity, form-wise variation, retrieval–generation linkage, object-type effects, layout-aware serialization, controlled OCR corruption, and bootstrap robustness were evaluated. The form-wise results further indicate that the OCR word error rate (OCR WER) alone does not fully characterize the downstream RAG performance. Overall, the proposed KV-ER Hybrid representation improved retrieval and answer generation under the evaluated OCR–RAG configuration without modifying the OCR model, retriever, or generator.