A study on the GraphRAG semantic retrieval algorithm for multimodal data

Abstract

To address the challenges of evidence chain breakage of vectors, collaborative constraint between image and structured fields is challenging, and the credibility of the generated results is insufficient with multimodal data, this paper proposes a GraphRAG semantic retrieval model for multimodal data. In order to realize the associative expression of text, image and structured data in the same retrieval link, based on the traditional RAG and knowledge graph enhanced retrieval, the model constructs a five-layer framework of data ingestion–semantic encoding–graph construction–retrieval enhancement–controlled generation. Text paragraphs, image target regions, and structured fields are encoded into modality-specific representations and subsequently projected into a unified 640-dimensional semantic retrieval space. Entity extraction, cross-modal entity linking, graph-path expansion, evidence aggregation, and credibility-aware re-ranking are then performed in this shared space to construct traceable multimodal evidence chains. A total of 18,640 technical texts and 7820 device images, 42,300 table records and 3600 retrieval questions were selected for verification in the experiment. The results showed that after GraphRAG enhancement, the median of Precision@5 increased from 0.812 to 0.869, the median of Recall @ 10 increased from 0.835 to 0.895, and the evidence hit rate increased from 84.1 to 89.7%. The Precision@5 and Recall @ 10 of the three-modal joint retrieval were 0.904 and 0.928, respectively, while the average end-to-end query time was 283 ms. Results demonstrate the ability of the model to increase the retrieval accuracy, evidence integrity and generation credibility in complex multimodal scenarios.