Retrieval-Augmented Multimodal Large Language Models for Visual Question Answering of Construction Occupational Health and Safety Hazards

A large language model-based question-answering system for crack information

Given the prevalence of crack defects in bridge structures and their associated risks to structural safety, enhancing the efficiency of crack analysis has emerged as a pivotal entry point for advancing intelligent bridge maintenance. To address the labor-intensive nature of manual processing, this paper proposes a systematic pipeline enabling natural language interaction with multimodal bridge crack data. First, modality-specific feature extraction was conducted within the pipeline: crack maximum widths were quantified from images using semantic segmentation and geometric algorithms, while critical semantic information was extracted from text reports via named entity recognition. To resolve the semantic dilution in tabular retrieval, a novel Multi-Perspective Table Linearization (MPTL) method was introduced to construct a high-density knowledge base. Ultimately, a Retrieval-Augmented Generation (RAG) technology invoking a large language model (LLM) was adopted to facilitate conversational crack analysis. Ablation results demonstrated that MPTL significantly surpassed baseline tabular integration methods in both recall rate and attribute matching accuracy, maintaining superior performance stability as the dataset scales changed. Additionally, efficiency assessments confirmed the system’s high reliability and scalability, with the RAG strategy successfully bounding the LLM’s context load independently of the underlying database capacity. Furthermore, the framework’s inherent robustness was discussed; specifically, synergizing MPTL with the contextual tolerance of both BriBERT and the LLM effectively mitigated the adverse impacts of noisy and incomplete inspection records. In summary, this research provides a highly accurate, scalable, and reliable framework for automated bridge defect analysis, offering a practical methodology to enhance data utilization in bridge management.

Retrieval-Augmented Large Language Models for Evidence-Based Hazard Log Generation in Emerging Aviation Systems

We introduce a retrieval-augmented synthesis pipeline for deriving structured hazard logs for emerging aviation concepts from historical aviation accident evidence. NTSB accident reports are transformed into a schema-consistent corpus combining coded findings and narrative mechanisms for semantic indexing. Mechanism-level retrieval uses sentence-transformer embeddings, a FAISS inner-product index, evidence-derived seed extraction, and maximal marginal relevance to obtain diversified, scenario-relevant cases. Hazard generation is constrained by strict JSON schema validation, one-to-one evidence binding, explicit causal sequencing, and enforced primary-mechanism uniqueness. A multipass strategy with critic-based filtering and deterministic de-duplication improves robustness against mechanism repetition and evidence drift. Evaluation of an urban eVTOL safety-landing scenario compares locally deployed open-weight models under identical constraints. Retrieval augmentation supports mechanism-specific and traceable hazard derivation compared to unconstrained scenario-based prompting. Mistral-7B requires multipass generation to achieve acceptable mechanism diversity and evidence consistency, whereas GPT-OSS-20b produces structurally valid and mechanism-differentiated hazard sets in a single pass. Scaling to GPT-OSS-120b yields only marginal improvements at substantially higher computational cost.

A Systematic Evaluation Framework for AI-Driven BIM Question Answering Systems

Large language models for evidence-based planning: Evaluating an SLR-RAG framework for knowledge synthesis of urban vacant land

Urban challenges necessitate robust evidence synthesis, yet Large Language Models (LLMs) applied to urban studies face factual inconsistencies and a lack of domain-specific grounding. This paper proposes and evaluates a framework for RAG-enhanced LLMs by integrating up-to-date knowledge and policy-relevant evidence into model outputs. Urban Vacant Land (UVL) is selected as the case study, and a systematic literature review (SLR) is conducted to build a retrieval literature base. The results indicate that RAG significantly improves the accuracy of LLMs under structured automatic evaluation. However, although retrieval augmentation provides models with access to domain-specific evidence, its benefits for open-ended planning questions are not consistently reflected in expert ratings, particularly for consistency and creativity. The limited improvement can be attributed to the characteristics of the planning questions, the mismatch between textual information and the spatial data required for urban planning in current RAG pipelines, and potentially ineffective prompting that fails to elicit deeper reasoning. This study contributes to the field by elucidating the capabilities and limitations of LLMs and RAG in urban studies, highlighting that while text-only retrieval is insufficient for context-rich analysis, future advancements in spatially aware hybrid retrieval offer a promising pathway forward.

Retrieval-Augmented Generation in LLMs for Mental Health: Quantifying the Incremental Contribution of Retrieval Within a Layered Safety Architecture

Digital mental health interventions (DMHIs) offer scalable support, but ensuring they accurately detect users'intent during volatile situations can be challenging. Pure parametric Large Language models (LLMs) do not contain specific safety critical architecture, and can miss critical cues, or hallucinate, undermining reliability. Retrieval Augmented Generation (RAG), which supplements an LLM with retrieved context, could enhance intent detection during volatile situations. Commercially available DMHIs typically combine multiple independent safety layers like rule-based filters, symbolic escalation protocols, and neural classification. The incremental contribution of any single layer, however, remains unquantified. This paper evaluates six LLM models within a DMHI called Wysa, via a controlled comparison of RAG-enabled versus RAG-disabled modes. Anonymized real and synthetic user-chatbot exchanges were annotated by a qualified clinical team against multi-class intent categories (e.g. self-harm, abuse, panic). The study computed classification accuracy, recall, precision and F1 scores against ground truth labels and tested differences for statistical significance. Performance was also examined by risk category and inter-model agreement. While RAG caused a rise in false alarms, the trade-off is consistent with safety-critical design principles that prioritize sensitivity, where flagged cases are routed to additional review rather than acted on directly. Overall, these findings support RAG as a promising approach to improve the accuracy, consistency and safety of LLM-driven DMHIs. Keywords: Digital Mental Health Intervention, Large Language Model, Retrieval Augmented Generation, Accuracy, Recall, Precision

Self-Reflective VLM with Multimodal Retrieval-Augmented Generation for Long-Context Construction Regulatory Document Comprehension

A deep comprehension of construction regulatory documents is essential for successful construction engineering and management. Although vision–language models (VLMs) hold tremendous potential for achieving human-level understanding, their practical deployment remains constrained by input size limitations. Even when integrated with retrieval-augmented generation (RAG), most efforts process visual elements commonly found in regulatory documents as pure texts and struggle to verify the trustworthiness of the retrieved supporting evidences, increasing vulnerability to hallucination. To address these drawbacks, we present a self-reflective multimodal RAG-assisted VLM pipeline that extends standard VLMs from single-pass document interpretation to multiround retrieval-grounded reasoning. Specifically, it augments existing VLMs with an external multimodal RAG mechanism, moving beyond text-only retrieval by jointly leveraging visual representations and semantic summaries to identify relevant evidence pages. Furthermore, an iterative self-reflection is incorporated to evaluate evidence sufficiency and trigger adaptive question refinement, thus enhancing response reliability. This pipeline was evaluated using a 794 question-answering pairs compiled from 12 divisions of a construction specification, covering three representative question types—information extraction, single-modal reasoning, and multimodal reasoning. Results showed substantial performance gains over traditional, single-model, and multimodal RAG-assisted baselines. More importantly, fine-grained analyses revealed that the proposed pipeline is particularly effective for complex engineering management scenarios—such as interpreting multimodal regulatory requirements, verifying compliance conditions embedded across late-document pages, and synthesizing information from text, tables, and figures. By enabling accurate, context-aware comprehension of lengthy construction regulatory documents without additional model training, the proposed pipeline offers a practical and generalizable tool for engineering management workflows. It has the potential to support compliance verification, reduce interpretation errors, and enhance the timeliness and quality of decision-making in construction management.