Retrieval-Augmented Large Language Models for Evidence-Based Hazard Log Generation in Emerging Aviation Systems

Abstract

We introduce a retrieval-augmented synthesis pipeline for deriving structured hazard logs for emerging aviation concepts from historical aviation accident evidence. NTSB accident reports are transformed into a schema-consistent corpus combining coded findings and narrative mechanisms for semantic indexing. Mechanism-level retrieval uses sentence-transformer embeddings, a FAISS inner-product index, evidence-derived seed extraction, and maximal marginal relevance to obtain diversified, scenario-relevant cases. Hazard generation is constrained by strict JSON schema validation, one-to-one evidence binding, explicit causal sequencing, and enforced primary-mechanism uniqueness. A multipass strategy with critic-based filtering and deterministic de-duplication improves robustness against mechanism repetition and evidence drift. Evaluation of an urban eVTOL safety-landing scenario compares locally deployed open-weight models under identical constraints. Retrieval augmentation supports mechanism-specific and traceable hazard derivation compared to unconstrained scenario-based prompting. Mistral-7B requires multipass generation to achieve acceptable mechanism diversity and evidence consistency, whereas GPT-OSS-20b produces structurally valid and mechanism-differentiated hazard sets in a single pass. Scaling to GPT-OSS-120b yields only marginal improvements at substantially higher computational cost.