Generating Machine-Processable Specifications from Natural Language Using Large Language Models in Automotive Commissioning and Testing

Abstract

In vehicle production, commissioning and testing processes of electric and electronic components are essential for value creation and quality assurance. The emergence of software-defined vehicles, however, leads to an increased scope and complexity of these processes as software functions depend on electric and electronic components for perception, execution, and processing tasks. In this context, this paper tackles a common challenge: Software that is deployed in vehicle production to implement commissioning and testing processes is developed upon specifications that define prerequisites, procedures, and target results in natural language. Therefore, extensive human interpretation and manual translation into executable code are needed being susceptible to errors as well as time-consuming. The large number of vehicle configurations and rapid changes in vehicle software further complicate the development of commissioning and testing software, particularly as verbose textual dependency descriptions risk impairing comprehensibility. Machine-processable specifications facilitating automated validation and code generation or direct execution could consequently ensure consistency, reduce manual effort, and accelerate the development process. For this purpose, we examine the processability of commissioning and testing specifications in natural language by proposing a pipeline designed to systematically transform these specifications into a machine-processable format. In particular, we introduce a unified schema that serves as an input format for the large language models tasked with the transformation. Subsequently, several large language models are evaluated in practical trials, based on their ability to translate commissioning and testing specifications into a machine-processable notation. In summary, this study aims to enable more efficient and data-driven software development based on textual requirements. This work offers valuable insights into the suitability and applicability of large language models within the planning of automotive commissioning and testing processes, targeting enhanced automation and efficiency.