A large language model-based question-answering system for crack information

Abstract

Given the prevalence of crack defects in bridge structures and their associated risks to structural safety, enhancing the efficiency of crack analysis has emerged as a pivotal entry point for advancing intelligent bridge maintenance. To address the labor-intensive nature of manual processing, this paper proposes a systematic pipeline enabling natural language interaction with multimodal bridge crack data. First, modality-specific feature extraction was conducted within the pipeline: crack maximum widths were quantified from images using semantic segmentation and geometric algorithms, while critical semantic information was extracted from text reports via named entity recognition. To resolve the semantic dilution in tabular retrieval, a novel Multi-Perspective Table Linearization (MPTL) method was introduced to construct a high-density knowledge base. Ultimately, a Retrieval-Augmented Generation (RAG) technology invoking a large language model (LLM) was adopted to facilitate conversational crack analysis. Ablation results demonstrated that MPTL significantly surpassed baseline tabular integration methods in both recall rate and attribute matching accuracy, maintaining superior performance stability as the dataset scales changed. Additionally, efficiency assessments confirmed the system’s high reliability and scalability, with the RAG strategy successfully bounding the LLM’s context load independently of the underlying database capacity. Furthermore, the framework’s inherent robustness was discussed; specifically, synergizing MPTL with the contextual tolerance of both BriBERT and the LLM effectively mitigated the adverse impacts of noisy and incomplete inspection records. In summary, this research provides a highly accurate, scalable, and reliable framework for automated bridge defect analysis, offering a practical methodology to enhance data utilization in bridge management.