In high-risk industrial settings, leveraging large language models (LLMs) for automated accident analysis and generating safety reports has emerged as an efficient workflow. However, this approach is fundamentally constrained by the models’ inherent knowledge limitations, frequently resulting in analyses that lack domain-specific understanding and regulatory alignment.
To tackle this issue, we introduce SafeRAG, a multistage retrieval-augmented framework for safety report generation. Specifically, the framework uses an entity-centric approach that prompts the LLMs to internally generate domain-specific knowledge. Concurrently, it performs a hierarchical retrieval of external regulations relevant to the accident at topic, concept, and context levels. To obtain well-structured reports, we leverage prompt engineering, integrating internal and external knowledge. Furthermore, a domain-expert persona is also assigned to help LLMs analyze accidents from a specific perspective. To evaluate our approach, we construct a data set from 10,818 accident-description/report pairs collected from real-world industry reports. Experiments show that SafeRAG substantially outperforms baseline LLMs on metrics that include bidirectional encoder representations from transformers (BERTScore) and bidirectional auto-regressive transformers (BARTScore), demonstrating the effectiveness of our approach.
Voluntary safety reports provide valuable information for identifying potential risks and improving safety management in civil aviation. However, these reports are often large in volume, unstructured in format, and rich in domain-specific terminology, making manual analysis costly, inefficient, and difficult to scale. To address these challenges, this paper proposes TMCAS, an efficient large language model-assisted topic modeling framework for civil aviation safety reports. The proposed framework combines domain-adapted text embeddings, density-based clustering, representative sampling, noise repair, and large language model-based topic generation. Specifically, a contrastive learning-based fine-tuning strategy is introduced to enhance the semantic representation of aviation safety texts. An HDBSCAN-based clustering and sampling mechanism is then designed to select representative reports and reduce the computational cost of large language model inference, while a noise-repair strategy is used to improve topic coverage. Finally, large language models are employed to generate interpretable sentence-level topic labels and descriptions. Experiments demonstrate that TMCAS achieves superior clustering and interpretability while substantially reducing inference cost compared with document-wise LLM baselines.
Improving flight safety with flight data requires not only accurate detection of risk events, but more importantly, clear interpretation of their underlying causes at the level of pilot control behavior. Existing explainable AI techniques, such as feature importance maps, often require considerable domain knowledge to translate them into operationally meaningful explanations. Large Language Models (LLMs), which excel at language reasoning, bring a promising solution to this issue. However, applying LLMs in this domain presents key challenges such as modal inconsistency, limited classification ability, scarcity of task-specific data for fine-tuning, and lack of domain knowledge. To overcome these challenges, we propose FlightLLM, a prior-guided semantic LLM-based approach for interpretable flight safety analysis. Specifically, we first perform feature engineering to address modal inconsistency, combining statistical descriptors with physically meaningful flight indicators. This representation is further processed by a Semantic Discretization module, which converts abstract numerical patterns into qualitative descriptions that are more compatible with language reasoning. In addition, since LLMs are not inherently strong classifiers, CatBoost is incorporated as a statistical expert, and its prediction results are injected into the prompt as prior guidance. A contrastive few-shot learning strategy is further adopted to compensate for limited data. Finally, we design structured prompts to embed aviation-specific knowledge into the inference process. Using hard landing, a representative risk event with complex causal mechanisms, as an anchor point, we evaluate FlightLLM on a dataset of 704 real-world A320 flight samples. Experimental results show that the proposed approach achieves competitive classification performance while generating direct and reasonable explanations for event causes.
Workflow generation in visual creation systems such as ComfyUI demands not only syntactic accuracy but also expert-level reasoning over modular compositions. Existing large language model (LLM) approaches often treat this as a direct text-to-JSON generation task, struggling with structural brittleness and lacking the experiential knowledge required for effective design. We argue that successful workflow generation requires modeling knowledge itself, including its structure, hierarchy, and reasoning dynamics. To this end, we propose a knowledge-centric framework that learns to invert, inject, and infer with knowledge across multiple abstraction levels. We first perform knowledge inversion to distill hierarchical representations, ranging from full pseudo-codes and skeletons to high-level strategies, from large collections of real-world workflows. We then conduct knowledge injection through supervised fine-tuning, teaching the model to reason from task descriptions to strategies and from strategies to executable structures. During inference, the model performs reversible reasoning to synthesize executable workflows, augmented by self-refinement for structural coherence. Extensive experiments demonstrate that our method produces workflows with richer node diversity, more coherent structures, and higher execution success rates than existing systems, establishing a new foundation for knowledge-driven, agentic workflow generation.
Zhendong Li, Lei Sun, Ruibo Ming et al.· 0 citations
Context: The quality of software requirements is a critical determinant of success in mission-critical and large-scale software systems. Requirements smells, which indicate potential quality issues in software requirements, such as imprecision, ambiguity, and poor verifiability, can significantly impact project outcomes. Despite their crucial role in software projects, software requirement documents often contain inconsistencies, ambiguities, and conflicts that are challenging to detect and manage manually. Objective: We aim to develop a novel approach using data representation and a large language model (LLM) based approach to improve the detection of conflict and ambiguity requirements smells. Method: We construct domain-specific vector databases from software requirements and domain knowledge documents, where vectors represent embedded requirements and contextual domain information. These databases are populated offline and utilized during inference time for semantic similarity-based retrieval and smell detection. We compare our domain-augmented generation based approach with traditional reasoning-based methods for performance evaluation using diverse retrievers. Additionally, we include zero-shot and one-shot prompt techniques. Results: Our experimental results demonstrate that our RAGbased approach detects requirements smells across multiple datasets using five LLMs: Claude 3.5 Sonnet, Claude 3.7 Sonnet, GPT-4o, o3-mini, and GPT-5. For conflict smell detection, compared to previous approaches on cross-datasets, these models have superior F1 scores, up from 0.566-0.804 to 0.676–0.857. The integration of domain knowledge showed mixed results, generally improving recall while decreasing precision and F1 scores. For ambiguity smell detection, the GPT-5 model achieved an F1 score of 0.783 compared to 0.762 of previous approaches. LLM-based approaches, besides GPT-4o, had better precision scores compared to previous approaches. Domain knowledge integration substantially increased the F1 score of the tested LLMs, with the most significant improvement observed in Claude 3.7 Sonnet, which rose from 0.526 to 0.677. In contrast, misleading or low-quality domain knowledge decreased F1 scores. Conclusions: The proposed RAG-based approach offers a promising automated solution for detecting requirements smells, demonstrating competitive performance compared to existing methods. Integrating domain knowledge enhances detection capabilities while highlighting the importance of balancing precision and recall.
We introduce a retrieval-augmented synthesis pipeline for deriving structured hazard logs for emerging aviation concepts from historical aviation accident evidence. NTSB accident reports are transformed into a schema-consistent corpus combining coded findings and narrative mechanisms for semantic indexing. Mechanism-level retrieval uses sentence-transformer embeddings, a FAISS inner-product index, evidence-derived seed extraction, and maximal marginal relevance to obtain diversified, scenario-relevant cases. Hazard generation is constrained by strict JSON schema validation, one-to-one evidence binding, explicit causal sequencing, and enforced primary-mechanism uniqueness. A multipass strategy with critic-based filtering and deterministic de-duplication improves robustness against mechanism repetition and evidence drift. Evaluation of an urban eVTOL safety-landing scenario compares locally deployed open-weight models under identical constraints. Retrieval augmentation supports mechanism-specific and traceable hazard derivation compared to unconstrained scenario-based prompting. Mistral-7B requires multipass generation to achieve acceptable mechanism diversity and evidence consistency, whereas GPT-OSS-20b produces structurally valid and mechanism-differentiated hazard sets in a single pass. Scaling to GPT-OSS-120b yields only marginal improvements at substantially higher computational cost.
Michael Schultz, Simon Göppel· International Conferences on...· 0 citations