AI-Driven Knowledge Externalisation: From Unstructured Documents to Structured Data Models

From Unstructured Procedural Text to Knowledge Graphs: Comparing LLM-Based Extraction Strategies

As experienced workers retire across industrialized economies, organizations risk losing procedural expertise that often remains tacit, undocumented or scattered across unstructured documents. Translating this knowledge into structured, machine-readable representations is difficult to scale, labor-intensive, and prone to inconsistency when done manually. This paper addresses the automated construction of knowledge graphs from natural language procedural descriptions, developing a generic approach for transforming unstructured expert knowledge into structured knowledge graphs that support downstream retrieval and question-answering applications. Three text-to-knowledge-graph approaches were designed, implemented, and systematically evaluated. The first employed a large general-purpose language model (Qwen3-32B) with a single-stage zero-shot extraction prompt, the second applied the same strategy using a smaller base model (Llama2-13B), and the third combined supervised fine-tuning of the smaller model on synthetic extraction data with a decomposed extraction architecture targeting one to two ontology elements per phase. All approaches were evaluated across six procedural descriptions spanning multiple technical domains, with chunk size, model temperature, and ontology detail as configuration parameters. Results were assessed for intrinsic quality and extrinsic fitness for use, measured by question-answering accuracy in a Graph-RAG application. The results demonstrate that extraction strategy is a more decisive factor than model capacity. The fine-tuned model achieved a Question Answering (QA) pass rate of 55.3%, compared to 46.8% for the large model and 33% for the small base model, outperforming the general-purpose model on both intrinsic quality metrics and downstream performance. Average node degree, duplication rate, and ontology simplicity emerge as the strongest predictors of retrieval performance. Shorter ontologies consistently outperformed richer ones, suggesting that ontology design should be driven by the intended downstream application rather than semantic completeness. Over-extraction of procedural steps did not degrade performance but instead improved retrieval robustness by providing additional entry points for graph traversal. The findings offer practical guidance for designing scalable, locally deployable knowledge graph construction pipelines for procedural texts under computational and confidentiality constraints.

Structure then Query: Enabling Precise Analytical Queries over Unstructured Documents

Experiments on three real-world datasets demonstrate that AnnoIndex consistently outperforms state-of-the-art baselines, achieving the highest average F1 score while maintaining robust performance on complex multi-hop join and progressive reasoning queries.

From Tacit Knowledge to Structured Documents: A Framework for Knowledge Elicitation

The Assistant-Scribe-Knowledge Checker (ASK) framework for generating structured specification documents through guided interviews is evaluated in a consulting-firm setting where consultants are required to produce project “return-of-experience” documents to capture reusable knowledge.

From Knowledge Retrieval to Execution: Designing Executable Knowledge Systems

Generative AI is changing the role of knowledge in organisations. Traditional knowledge management (KM) systems have primarily supported storage, access and retrieval, assuming that knowledge is interpreted and applied by human users. In AI-enabled environments, however, organisational knowledge increasingly becomes a direct input into execution, shaping generated proposals, analyses, summaries, recommendations and other workflow outputs. This shift exposes a limitation of retrieval-oriented KM: fragmented, outdated or weakly governed knowledge can be amplified through AI-generated outputs, reducing consistency, reliability and trust. This paper introduces executable knowledge systems as a conceptual model for structuring organisational knowledge to support reliable human and AI-assisted execution. The term executable is used in a socio-technical sense. Knowledge does not necessarily become code, but is curated, validated and embedded into workflows so that it can guide outputs, decisions and actions. The paper distinguishes this concept from prior work on executable knowledge graphs and executable knowledge bases, which primarily focus on deterministic execution through rules, scripts or formalised representations. The paper further develops a framework of decay and compounding loops to explain how AI-enabled knowledge systems evolve over time. In decay loops, AI-generated outputs re-enter the knowledge environment without sufficient validation, allowing inconsistency and low-quality knowledge to accumulate. In compounding loops, curated knowledge assets are refined through governed feedback, domain ownership and controlled reuse, enabling improvements in reliability over time. The framework is informed by an exploratory case study within a global professional services organisation, where a curated knowledge environment was introduced to support AI-assisted workflows in the Retail, Consumer Products, Travel and Transportation domain. The evaluation compared outputs generated from a controlled, subject matter expert (SME)-validated knowledge dataset with outputs generated from an unconstrained organisational knowledge base. Findings indicate improved retrieval relevance and output quality when AI systems operate on validated knowledge assets. The paper contributes to KM research by reframing KM as a system design challenge for AI-enabled execution and by positioning governance, validation and feedback control as central mechanisms for reliable organisational knowledge use.

AI-Guided Metadata Construction for Meaning-Driven Digital Knowledge Systems: A Framework for Automated Metadata Generation and Semantic Discovery

Integrating Large Language Models (LLMs) with Oracle 26AI for Advanced Enterprise Analytics and Knowledge Management

Results indicate that LLM-augmented analytics on a converged Oracle 26AI platform can reduce average analytical query resolution time by approximately 60 percent relative to traditional BI report cycles, achieve semantic retrieval precision above 90 percent for enterprise knowledge corpora, and reduce generative output hallucination rates by more than half when grounding is enforced through in-database retrieval.