Construction of multimodal oracle bone semantic reasoning and knowledge graphs in archaeology under artificial intelligence technology

Abstract

To address the difficulty of establishing continuous associations among oracle bone script images, transcriptions, and archaeological knowledge, this study developed the Oracle-Artifact Multimodal Reasoning (OAMR) multimodal semantic reasoning framework. Vision Transformer (ViT)-B/16 and Bidirectional Encoder Representations from Transformers (BERT)-base-Chinese were used for image-text alignment. Prompt-constrained entity and relation extraction identified six entity types and eight relation types. Cross-source node calibration reduced the fragmentation of synonymous entities. The graph neural network integrated path probabilities and the confidence of the weakest edge to generate traceable reasoning chains. The experiments included 12,000 valid image-text pairs, which were divided into training, validation, and test sets at a ratio of 8:1:1. All models used the same data boundaries and were independently run five times. OAMR achieved an R@1 of 69.8% ± 0.6%. The F1 scores for entity recognition and relation extraction reached 92.4% ± 0.3% and 89.5% ± 0.4%, respectively. The reasoning accuracy reached 92.8% ± 0.4%. Compared with the best-performing baselines for each task, the differences in the core metrics were statistically significant after Holm correction. The results demonstrated that continuous integration of cross-modal representations, structured knowledge extraction, and confidence-aware path reasoning improved the accuracy and traceability of oracle bone script knowledge organization and semantic inference.