Skip to content
Open access

Key Technologies and Applications of Multimodal Cognitive Agent Construction in the Vertical Domain of Inspection and Testing

Jul 2026 · Frontiers in Computing and Intelligent Systems · 0 citations · 30 references

Abstract

 Inspection and testing underpin product safety and regulatory compliance across industries such as manufacturing, healthcare and food and beverage. However, conventional engineering test processes that mainly depend on manual, offline actions, failed to adapt to next generation, automated, digital and environmental-sensitive testing, resulting in very low productivity and high cost for most firms. The automation and digitalization of inspection and testing processes have become a research hotspot in both academia and industry. The natural language processing and computer interpretation of test results have been a focus of AI research as well. Nonetheless, due to the lack of real data, the verification and simulation of real inspection and testing environments are still difficult for researchers. As a solution, we manage to develop a multimodal cognitive test system that fuses the textual regulatory documents and instrument data, through a series of modules and processing flows. Our test system is based on a so-called multimodal cognitive agent, which includes large language model, vision module, knowledge graph and retrieval-augmented generation. We introduce the design, development and application of our test system, which used for rubber heater in a glass factory, and layout the future challenges for the exploration of multimodal agent technology in the test engineering. The project is supported by the Guangxi Key Research and Development Program. This paper was partially presented at the IEEE 2023 International Conference on Intelligent Commerce (ICIC).

Read PDF

Similar papers

Conference 2026

A Cognitive Framework for Pressure Equipment Inspection Based on Multi-Agent AI Systems and Vllm Models

: This paper investigates the transformation of the traditional pressure equipment (PE) inspection process into an intelligent digital workflow through the application of advanced artificial intelligence technologies. The research focuses on overcoming the issues of inspector cognitive overload and the inefficient management of extensive documentation during the interpretation of the Pressure Equipment Directive (PED) and national regulations. An innovative multimodal architecture is proposed, based on a Multi-Agent System (MAS) and Vision-Large Language Models (VLLMs) utilizing a Retrieval-Augmented Generation (RAG) mechanism for dynamic compliance validation. Through a case study of a Liquefied Petroleum Gas (LPG) tank inspection, it is demonstrated how specialized AI agents—including agents for visual analysis, technical diagnostics, compliance verification, and interactive communication—can autonomously identify defects, calculate the remaining service life, and generate valid reports for registries such as CROPP.

Aleksandar Cvetić, Angelina Njegus · 0 citations
Preprint Jul 2026

An Exploration of Agentic Information Fusion for Test Maintenance Prediction

Test maintenance is a critical, yet costly, activity - particularly as codebases rapidly evolve. To assist, we present MAST, a multi-agent framework that predicts which test cases require maintenance following changes to the production code. This identification task is necessary as a precondition to any subsequent maintenance activities, but remains challenging due to the complex relationships between production and test code. MAST advances the state-of-the-art by integrating multiple analyses -- including static, lexical, and semantic analyses - through an intelligent fusion and post-check procedure and by focusing on a realistic use and evaluation setting - i.e., standardized input formats, repository-level analyses, and the ability to infer relations between test and production artifacts rather than assuming a pre-existing mapping. We evaluated MAST on 21 industrial Java repositories from Ericsson AB, considering situations where test maintenance both was and was not required in the ground truth. MAST yielded superior precision to a state-of-the-art baseline - resulting in a higher accuracy, F1, and F2 score - with only some loss in recall. Our ablation study demonstrates the value of each analysis in producing the final recommendations. MAST illustrates the potential of multi-agent systems that can fuse multiple information sources when performing software testing tasks.

Jingxiong Liu, N. Mohammadiha, Gregory Gay · 0 citations
Review Open access 2026

Large AI Models Empowering Intelligent Manufacturing: Architecture, Evolution, and Prospects

Large AI models are reshaping intelligent manufacturing from isolated automation toward knowledge-intensive, model-assisted production systems. Yet their industrial value depends not on model scale alone, but on how language, vision, code, sensor data, engineering knowledge, and feedback mechanisms are integrated into deployable manufacturing workflows. This review examines recent progress in large AI models for intelligent manufacturing, covering model architectures, adaptation strategies, system integration, and applications across product development, production processes, equipment maintenance, and manufacturing services. A lifecycle-based framework is used to organize the literature and distinguish model capabilities from the data resources, retrieval mechanisms, simulation and optimization tools, digital twins, edge-cloud infrastructure, and human validation required for deployment. Current evidence suggests that large models show more reliable value in bounded, information-rich tasks, whereas safety-critical control and production-scale autonomy remain insufficiently validated. The review further summarizes challenges in data quality, domain adaptation, reliability, interpretability, latency, cybersecurity, cost, benchmarking, and responsibility allocation. By linking application scenarios, system-level enablers, and evidence maturity, this review provides a structured perspective for assessing the practical value of large AI models in manufacturing.

Baotong Chen, Lu Dai, Chuangjian Wang et al. · 0 citations
Review Jul 2026

Multi-Agent LLM Collaboration for Unit Test Generation via Human-Testing-Inspired Workflows

Recently, the emergence of Large Language Models (LLMs) has spurred a surge of research into automated unit test generation, yielding impressive performance and reducing manual effort. However, existing LLM-based approaches still suffer from two major limitations: (1) they follow rigid, procedural workflows that underutilize the autonomous reasoning potential of LLMs, making it difficult to dynamically adapt testing strategies based on real-time feedback; and (2) they rely on rule-based context extraction that is not tailored to test generation, failing to capture fine-grained code dependencies and test-specific knowledge required for deriving test requirements. In this paper, we propose TestAgent, an LLM-based test generation approach that addresses the above limitations by emulating human testing practices via a multi-agent collaboration mechanism. Particularly, TestAgent designs three specialized agents, namely a requirement planner, a test generator, and a test reviewer, to simulate how developers understand, construct, and validate unit tests. To unleash the autonomous capabilities of LLMs, we equip TestAgent with a set of tool APIs that can be invoked dynamically in an on-demand and adaptive manner. To further support repository-level reasoning, TestAgent constructs a test-specialized knowledge graph via static analysis, which captures code entities and their dependencies across the project and persistently stores testing artifacts (e.g., test reports and failure analyses) produced during generation. Experimental results show that TestAgent achieves 97.46% execution rate, 92.34% line coverage, 90.24% branch coverage, and 83.69% mutation score on six Java projects, outperforming LLM-based baselines across all metrics and achieving substantially higher mutation scores than search-based tools.

Quanjun Zhang, Ye Shang, Siqi Gu et al. · 0 citations
Jul 2026

Generating Machine-Processable Specifications from Natural Language Using Large Language Models in Automotive Commissioning and Testing

In vehicle production, commissioning and testing processes of electric and electronic components are essential for value creation and quality assurance. The emergence of software-defined vehicles, however, leads to an increased scope and complexity of these processes as software functions depend on electric and electronic components for perception, execution, and processing tasks. In this context, this paper tackles a common challenge: Software that is deployed in vehicle production to implement commissioning and testing processes is developed upon specifications that define prerequisites, procedures, and target results in natural language. Therefore, extensive human interpretation and manual translation into executable code are needed being susceptible to errors as well as time-consuming. The large number of vehicle configurations and rapid changes in vehicle software further complicate the development of commissioning and testing software, particularly as verbose textual dependency descriptions risk impairing comprehensibility. Machine-processable specifications facilitating automated validation and code generation or direct execution could consequently ensure consistency, reduce manual effort, and accelerate the development process. For this purpose, we examine the processability of commissioning and testing specifications in natural language by proposing a pipeline designed to systematically transform these specifications into a machine-processable format. In particular, we introduce a unified schema that serves as an input format for the large language models tasked with the transformation. Subsequently, several large language models are evaluated in practical trials, based on their ability to translate commissioning and testing specifications into a machine-processable notation. In summary, this study aims to enable more efficient and data-driven software development based on textual requirements. This work offers valuable insights into the suitability and applicability of large language models within the planning of automotive commissioning and testing processes, targeting enhanced automation and efficiency.

Katja Köhler, Aiman El Asad, Michael Hahn et al. · 0 citations
Review Open access Jul 2026

Large language models in intelligent manufacturing and mechanical engineering: a review of robotics, fault diagnosis, design, and engineering knowledge workflows

Large language models (LLMs) are attracting growing attention in robotics, mechanical engineering, and mechatronic systems. This review shows that, in most engineering settings, their value is not that they replace simulation, control, or numerical analysis tools, but that they help engineers work across documents, data sources, software environments, and natural-language instructions more efficiently. The review surveys recent studies across robotics and embodied systems, fault diagnosis and maintenance, design and simulation, industrial knowledge workflows, manufacturing knowledge systems, and selected adjacent engineering applications only where they provide transferable methodological insight for intelligent manufacturing. Across these areas, the evidence shows a movement away from prompt-only demonstrations and toward retrieval-augmented generation (RAG), multimodal, tool-connected, and agent-based systems that are more tightly grounded in engineering evidence and operational context. The review further shows that the strongest results generally come from hybrid architectures that combine LLMs with RAG, knowledge graphs, multimodal perception, digital twins, validation modules, or downstream engineering tools. Even so, important limitations remain, including weak grounding in cluttered or ambiguous settings, limited numerical and spatial reliability, poor long-horizon robustness, fragmented benchmarks, and incomplete integration with trusted engineering software. Overall, the evidence indicates that LLMs are becoming useful semantic and coordination layers in engineering workflows, but not dependable engineering substitutes. Their most credible near-term role is in human-in-the-loop, evidence-grounded systems where retrieval, validation, tool use, and structured knowledge help keep outputs useful and bounded in safety-relevant tasks.

Sherif Samy Sorour, Anwar Sahbel · 0 citations