Assessing Graph Data Serialization and Context Selection for Knowledge-Augmented Question Answering with Large Language Models

Abstract

Large language models (LLMs) can produce inaccurate answers even when relevant facts are supplied in the prompt. This study examines how graph data serialization, entity identifiers, and context selection affect factual question answering in knowledge-augmented prompting. Four synthetic knowledge graph datasets containing 6, 10, 14, and 20 people entities were constructed using the familiar FOAF ontology, which was expanded with custom properties. Eight separate serialization formats were evaluated: natural language, XML, JSON, and RDF/XML and their variants. These formats were combined with two separate LLMs: Llama-3.2-3B-Instruct and Qwen2.5-3B-Instruct. Data was supplied through system prompts without training or fine-tuning the models. Three experimental stages examined complete graph contexts, answers obtained using predefined SPARQL queries in AllegroGraph, and descriptions of individual entities. In total, 19,044 model responses were evaluated in the experiment. Results indicate a clear accuracy–complexity trade-off: replacing natural language with XML using full-name identifiers leads to an increase in accuracy, from 54.50% to 89.17% for Llama and from 72.33% to 82.67% for Qwen when considering the 10-person-entity dataset. The findings indicate that optimal LLM reasoning requires a balance between structural rigidity and readability; using explicit, human-readable identifiers instead of numeric abstractions improves context retention and minimizes errors. However, this approach demands caution, as it can lead to up to a 29.72% relative increase in token consumption compared with using natural language. Ultimately, improving LLM information recall requires a carefully calibrated balance between symbolic representation, semantic expressiveness, and knowledge scalability.