Application of Large Language Models for Detecting Semantic Ambiguity in Industrial Instructions: Impact on Human–Machine Interaction and User Experience in Process Automation Systems of a Metallurgical Plant
A method called Semantic Latent Choice Detection is presented, designed to systematically identify interpretation ambiguities in process instructions and operator commands, and shows a statistically significant reduction in errors related to misinterpretation of process regulations.
Abstract
In the context of industrial digitalization and the widespread adoption of process automation systems, Knowledge Management Systems (KMS) play a key role in providing operational personnel with up-to-date instructions and regulations. However, the inherent ambiguity of natural language in technical documentation remains a serious obstacle, leading to incorrect operator actions, process deviations, and increased safety risks. This article investigates the integration of Large Language Models (LLMs) into KMS and its impact on user experience and human–machine interaction in industrial automation environments. A method called Semantic Latent Choice Detection is presented, designed to systematically identify interpretation ambiguities in process instructions and operator commands. Unlike existing approaches that require access to the internal model architecture (“white box”) or token-level logits, the proposed method is logit-free and operates with closed commercial LLMs (“black box”) via standard API interfaces. The method analyzes the semantic similarity of binary text blocks and polysemous terms within the context of a specific technological process. Using a metallurgical production case study, we demonstrate how the system detects hidden semantic collisions (e.g., the difference between “adding ferroalloys into the ladle” and “feeding ferroalloys onto the conveyor”) that are missed by traditional rule-based validation methods. Instead of arbitrarily selecting an interpretation, the system initiates a clarification request to the human operator, thereby reducing cognitive load, preventing erroneous automated decisions, and increasing trust in the KMS. An empirical evaluation conducted in a real-world industrial setting (unit control rooms and dispatch centers) shows a statistically significant reduction in errors related to misinterpretation of process regulations. The article contributes to the fields of automation engineering, knowledge management, and human-centered automation by proposing a novel method for validating operational instructions in high-risk industrial environments.
In fast-evolving software systems, effective 'natural language requirements parsing' and downstream change effect analysis capability across a multitude of codes represents low-hanging-fruit in this regard. We present a structured framework to deploy Large Language Models (LLMs) for automating two essential software engineering tasks, namely requirement interpretation and change impact analysis Utilizing the inherent understanding of semantics offered by transformer-based LLMs, the novel approach advances by converting vague and unstructured requirement documents into structured but machine-readable specifications to offer a direct traceability mapping from requirements to system components. Additionally, the framework leverages LLM-driven dependency analysis to predict and quantify how change effects percolate through connected modules which can minimize manual effort and human errors. This approach combines prompt engineering and retrieval-augmented generation (RAG) for domain-relevant accuracy plus fine-tuning techniques. On open-source and enterprise-grade software projects, experimental evaluations show that disambiguation accuracy, traceability precision, and change impact coverage of our approach are orders of magnitude better than state-of-the-art rule-based or static analysis tools. Notes: The results illustrate the application of LLMs at scale and demonstrate how these can alter software engineering workflows by removing bottlenecks (at a massive scale) at different stages of the software development lifecycle. In this research, we provide a generalizable pipeline that helps to bridge the gap from NLP advancements into practice for software lifecycle management.
Nithya Krishnan, Kumaran Ramanujam, Suresh Babu Narra et al.· 2026 International Conferenc...· 0 citations
The article is devoted to the theoretical and methodological substantiation of the concept of intelligent automation of economic processes based on the integration of Retrieval-Augmented Generation (RAG), Large Language Models (LLM), Prompt Engineering, and Automatic Engineering technologies. The modern digital economy is moving from technical automation to cognitive automation, in which self-learning systems are emerging that can adapt to environmental changes, analyze the results of their own activities, and generate new economic solutions. RAG acts as a cognitive intermediary between generation and data retrieval, ensuring the factual reliability of analytical results. LLMs provide semantic interpretation of information and create conditions for natural-language scenario modeling. Prompt Engineering determines the quality of interaction between humans and the system by transforming users’ analytical intentions into a formalized query structure. Particular attention is paid to Automatic Engineering as a meta-level of cognitive management that ensures automated prompt improvement, reconfiguration of generation parameters, development of decision metamodels, and formation of digital twins of management processes. A multilevel cognitive-engineering model of economic management is proposed, which includes strategic, cognitive-technical, self-learning, and reflexive levels. This architecture forms a closed cognitive cycle of “generation – evaluation – optimization – updating,” which ensures the system’s capacity for reflexive self-learning and evolutionary development. The practical significance of the research lies in creating a methodological basis for implementing agentic AI solutions in strategic planning, risk forecasting, and enhancing the intellectual resilience of economic systems in the digital economy.
Keywords: intelligent automation, Retrieval-Augmented Generation (RAG), Large Language Models (LLM), Prompt Engineering, Automatic Engineering, cognitive architecture, digital economy, strategic management, self-learning economy, artificial intelligence.
S. Arefiev, Serhii Hildi· Ukrainian Journal of Applied...· 0 citations
In agile development, user stories express stakeholder needs, and the associated acceptance criteria (AC), written in the Given/When/Then (GWT) notation, specify the behaviour expected of the system under stated preconditions. Their manual translation into UML activity diagrams is laborious and remains sensitive to wording differences between authors. Existing automated approaches rely on lexical matching or on grammatical rules. Both fail when an identical precondition is expressed through different wording, a case that occurs frequently in collaborative backlogs. This study introduces GTA (GWT-to- Activity), a deterministic pipeline that converts GWT AC into UML activity diagrams. The clause structure is first recovered through dependency parsing. Paraphrased Given states are then merged based on sentence embedding similarity using Sentence-BERT (SBERT), and natural language inference (NLI) is used to separate semantic opposition from lexical variation, with thresholds calibrated on annotated data to preserve output stability under paraphrastic variation. Eleven behavioural patterns are supported, from simple sequences to nested compound conditions. The pipeline was evaluated on 107 test sets containing 230 AC drawn from four heterogeneous corpora of synthetic, open-source, and benchmark origin, and it reaches a node F1-score of 99.26%, an edge F1-score of 97.37%, and an exact topological match rate of 95.33%. All eleven patterns are recovered, and every generated file is a valid PlantUML that requires no post-processing. The output is structurally correct, deterministic from one run to another, and traceable to the thresholds set at each gate.
Samia Nasiri, Salim Bloundi, M. Lahmer· International Journal of Adv...· 0 citations
The adoption of large language models (LLMs) in software engineering has enabled the potential to automate complex activities such as requirements analysis. This paper presents an empirical performance analysis of four modern LLMs: GPT-4o, Aya, Gemma and Phi-4 on the task of automated classification of atomic software requirements. The LLMs were invoked under two different scenarios to solve a multilabel classification of 296 requirements extracted from the PROMISE[Formula: see text] dataset. The baseline scenario relies solely on internal model knowledge, whereas the rubric-augmented scenario uses formal definitions derived from the SQuaRE product quality model. The results indicate that GPT-4o consistently attains the highest overall classification accuracy under both scenarios. Moreover, all LLMs exhibit strong and stable performance in identifying functional, performance efficiency and security requirements. Inter-rater agreement assessments using Cohen’s and Fleiss’ Kappa coefficients further demonstrate moderate to substantial agreement among the outputs of the evaluated LLMs.
Nourchène Elleuch Ben Ayed, Jaber Jemai, Keletso J. Letsholo et al.· Journal of Information &...· 0 citations
The results indicate that current general-purpose LLMs can achieve practically significant performance on the unstructured NL-to-LTL task without task-specific fine-tuning, and suggest that modern LLMs are becoming viable front-end assistants for semi-automated formalization workflows.
This evidence-centered structured survey synthesizes representative work available through May 31, 2026 across software engineering tasks, software security tasks, adaptation mechanisms, artifact granularity, and evaluation design and introduces an assurance framework that separates functional correctness, security, operational reliability, evidence provenance, and agent authority.
Wei Lin, Tao Zhou, Zhaofei Xie et al.· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.
