Structured Knowledge Graph for Scholarly Exploration: IR and QA System for ETDs and their Elements

Abstract

Electronic Theses and Dissertations (ETDs) contain rich and detailed scholarly knowledge, but accessing that remains difficult. Conventional search systems largely operate at the document level and provide only coarse access to long-form academic writing. Though large language models (LLMs) can generate fluent responses, those often lack reliable grounding in the internal structure and semantic relationships of ETDs. We address these challenges with a knowledge graph (KG) framework for scholarly exploration and LLM-aided question answering (QA) over ETDs and their elements. The proposed approach models ETDs using a formal ontology and constructs an ETD-specific KG that captures relationships among documents, topics, chapters, figures, captions, and other fine-grained elements. Building on this representation, we introduce an ontology-guided hybrid retrieval framework that combines element-level textual retrieval, graph-based evidence retrieval, and LLM-based answer generation. By integrating fine-grained indexes with structured knowledge representations, the system supports both direct answers to focused questions and exploratory, relationship-aware access to ETD collections, enabling queries over documents as well as their internal semantic and structural components. Our QA user study with 47 participants from four engineering departments compares: plain retrieval, retrieval with LLM-based answer generation, and the proposed KG–guided framework. The proposed method achieves 40% higher overall user satisfaction than the retrieval-only baseline, with improvements of 53% in readability and 48% in clarity. It outperforms competing methods on the user-valued dimensions of informativeness and clarity, with gains observed for 87% of participants. These findings for ETDs confirm that combining fine-grained retrieval, structured KGs, and ontology-guided retrieval improves long document scholarly exploration and QA. Code, data, and user study results can be found here1.