This work presents an end-to-end, deployment-aware testing pipeline for IoT-based automotive applications that combines requirement-driven test and code generation with large language model (LLM) and vision-language model (VLM) assistance, and human-in-the-loop curation to reduce manual effort and improve consistency.
Abstract
Testing embedded software in modern vehicles is challenging due to system complexity, decentralized architectures, and strict safety and performance constraints. In this work, we present an end-to-end, deployment-aware testing pipeline for IoT-based automotive applications. The pipeline combines requirement-driven test and code generation with large language model (LLM) and vision-language model (VLM) assistance, and human-in-the-loop curation to reduce manual effort and improve consistency. Using Eclipse openDuT, it supports flexible, distributed deployment across geographically separated cyber-physical and IoT infrastructures, optimizing for node availability and cross-organizational coordination. For validation, we conduct a case study using a Child Presence Detection System (CPDS), achieving full functional requirement coverage across all 9 requirements and 100% Gherkin generation accuracy on the controlled requirement set. Distributed test execution across geographically separated ECUs via Eclipse openDuT confirms the pipeline's applicability to OEM--supplier testing workflows.
The increasing complexity of modern software-intensive systems, particularly in the automotive domain, demands new approaches to bridge the gap between high-level engineering specifications and executable, safety-compliant code. This need is amplified by the rapid transition toward software-defined vehicles, where highly dynamic, updateable software functions significantly enlarge the scope and frequency of engineering activities and require scalable, transparent, and adaptive development processes. While recent advances in Large Language Models have demonstrated strong capabilities in automating tasks such as requirements analysis, code generation, and documentation, their deployment in safety-critical engineering workflows remains challenging due to the need for transparency, traceability, and controlled decision-making.This paper presents a modular multi-agent Large Language Model (LLM) pipeline that automates key steps of the systems engineering lifecycle - from requirement structuring and compliance checking to code and test generation - using specialized LLM agents orchestrated within a unified architecture. A central contribution of this work is the integration of a Human-in-the-Loop subsystem, which introduces configurable review checkpoints at critical stages such as requirements analysis, compliance assessment, code generation, and test creation. The human-in-the-loop module enables engineers to approve, reject, or modify intermediate results, ensuring human oversight, enhancing trustworthiness, and enabling adherence to functional safety standards.The system supports heterogeneous input formats and provides end-to-end traceability through structured outputs and detailed monitoring of performance metrics including model usage, token consumption, and automation efficiency. Initial evaluations indicate that the combination of multi-agent specialization and human-in-the-loop-guided oversight can significantly reduce engineering effort while maintaining the transparency and reliability required for regulated domains.By embedding controllable human supervision into the LLM-driven pipeline, this work offers a practical and scalable architecture for integrating Artificial Intelligence (AI) automation into safety-critical systems engineering processes, with particular relevance to automotive software development.
Marcel Padubrin, A. Kulzer, Erol Guerocak· SAE technical paper series· 0 citations
This paper introduces the Sensor Traffic Generator (STGen), a lightweight, pure-software testbed for evaluating IoT application- and transport-layer protocols at scale. Relative to existing software-based IoT evaluation platforms, STGen combines three design decisions that, to the best of our knowledge, no prior testbed offers together. Every emulated sensor node runs as an independent operating-system process using a real transport stack rather than a discrete-event model or a container. Sensor workloads are generated using physically grounded stochastic models calibrated against real deployment data. Experiments are specified in three independent tiers, IoT Protocols (N), Scenarios (M), and Networks (L), reducing configuration effort from a combinatorial N×M×L problem to a linear N+M+L workflow, with new protocols integrated by overriding a four-method abstract interface. STGen operates above OSI Layer 4 and therefore does not model PHY- or MAC-layer behavior, such as RF interference, CSMA/CA collision avoidance, or duty cycling. The sensor models are calibrated using 1,826,223 real-world readings from the Intel Berkeley Research Laboratory; for temperature, the synthetic stream matches the 37-day measurements of 54 Mica2Dot motes with a Kolmogorov–Smirnov D of 0.071 and a Jensen–Shannon divergence of 0.018, showing that STGen reproduces the statistical structure of real sensor data rather than only plausible values. By inverting these calibrated models, STGen also synthesizes labeled false-data-injection anomalies that are separable from normal traffic, with a receiver operating characteristic AUC of 0.898 for stealthy drift and 1.0 for hard physical range violations. In our experiments, STGen instantiates 6000 concurrently emulated sensor nodes on a commodity workstation in 1.02 s using 0.62 GB of memory (approximately 99 KB per node), which is more than two orders of magnitude below the per-node memory costs of container- and VM-based testbeds. STGen also exposes deployment-relevant behavior that controlled emulation alone may hide. Under live wide-area jitter, MQTT and CoAP exhibit different loss and latency patterns than those observed under uniformly degraded NetEm conditions, including MQTT reconnection storms. These results show that STGen provides a scalable and reproducible bridge between lightweight protocol emulation and practical deployment-oriented IoT protocol evaluation.
H. Islam, M. R. Maharaz, M. Georgiades et al.· Journal of Sensor and Actuat...· 0 citations
As the automotive industry transitions toward Software-Defined Vehicle (SDV), Over-the-Air (OTA) updates have become a critical capability. However, conventional update mechanisms often struggle to ensure update success due to the gap between static regulatory compliance and the dynamic operational complexities of modern vehicle architectures. This paper proposes a dependency-aware OTA orchestration framework that addresses these challenges by integrating four operational dimensions—Dependency, Safety, Update Sensitivity, and Governance—into a graph-based scheduling logic. Designed to align with major industrial standards such as AUTOSAR, UNECE R155/R156, and ISO 26262, our framework provides a standardized yet flexible foundation for managing software components across multiple Electronic Control Units (ECUs) environments. Evaluated against a Conventional Sequential Baseline that represents current industrial practices, the proposed approach demonstrates superior performance across three Research Questions (RQs): effectiveness in improving update success rates, efficiency in execution time and update requests through optimized scheduling, and feasibility in maintaining system-wide integrity by successfully reconciling stringent safety requirements and diverse update sensitivity constraints. These findings offer actionable insights for implementing high-integrity OTA solutions that meet both functional safety and legal requirements in production-grade SDV environments.
Juyeon Park, In-Young Ko· SIGSOFT FSE Companion· 0 citations
IoT software systems involve implicit and explicit interactions among humans, devices, and environments, which makes requirements specification and evaluation more complex and context-sensitive. Large Language Models (LLMs) emerge as potential tools to support Requirements Engineering (RE) activities, although empirical evidence on their application in IoT software systems remains limited. In this article, we analyze how Generative Artificial Intelligence (GenAI) tools can support the specification and quality evaluation of requirements in IoT software systems through a two-stage application study. First, we carried out a Rapid Review of the Scopus database to identify GenAI tools used in RE. Next, we conducted an experimental study in a real-world setting to develop a smart home system to support a person with physical disabilities. We used ChatGPT to generate (i) requirements from a vision document and a predefined specification structure, and (ii) the evaluation of the requirements against ISO/IEC/IEEE 29148 criteria. The model generated an initial specification containing 35 requirements (23 functional and 12 non-functional) and provided suggestions for improvement during the quality evaluation phase. Only a subset of the requirements required additional adjustments by the engineers. The results indicate that GenAI tools have the potential to support the initial generation and review of requirements, reducing the initial documentation effort. However, supervision by requirements engineers remains essential. As an additional contribution, this study proposes the SpecEval-IoT environment to automate the specification and evaluation of requirements for IoT software systems.
S. Souza, Káthia Marçal de Oliveira, C. M. D. Farias et al.· Engineering Interactive Comp...· 0 citations
With the increasing complexity of the "cloud–pipe–end" architecture in intelligent connected vehicles (ICVs), the coexistence of multiple communication protocols such as automotive Ethernet, CAN bus, and V2X communication, coupled with the successive release of regulations and standards including UN R155, ISO/SAE 21434, and GB44495, the traditional manual approach to writing cybersecurity test cases faces severe challenges in terms of efficiency, coverage, and compliance. Large language models (LLMs), with their powerful semantic understanding and content generation capabilities, offer a new technical pathway for the automated generation of test cases. However, general-purpose LLMs lack domain-specific knowledge of automotive cybersecurity, and their direct application encounters challenges such as knowledge deficiency, insufficient scenario adaptation, and untrustworthy generated content. This paper proposes a methodological framework for automatic test case generation in ICV cybersecurity testing, named AutoCarSec. Centered on domain knowledge injection, the framework adopts a three-layer progressive architecture consisting of a knowledge enhancement layer, a scenario modeling and generation layer, and a multi-agent collaborative execution layer, systematically addressing the three core issues of knowledge acquisition, scenario modeling, and quality assurance for LLMs in the automotive cybersecurity testing domain. This paper elaborates on the detailed design of four key stages: structured construction of automotive cybersecurity domain knowledge, automatic test scenario modeling based on attack trees and TARA, LLM-driven hierarchical test case generation, and multi-agent collaborative verification with iterative optimization. It provides a logically sound and highly operable methodological reference for the intelligent transformation of cybersecurity testing for intelligent connected vehicles.
Software complexity in automotive has increased fourfold since 2010, yet productivity has remained flat. This growing gap threatens automakers’ ability to innovate while spending on automotive software continues to climb, over $100billion in recent years, with projections to double every 7 to 8 years. The shift to software defined Vehicles addresses this through hardware consolidation replacing dozens of distributed ECUs with centralized computers that enable software virtualization and hardware to software decoupling. This transformation brings new challenges such as managing ML models across fragmented paltforms, diverse hardware configuration, limited edge computing resources, intermittent connectivity, and strict safety requirements under ISO 26262 and SOTIF. Traditional MLOps practices don’t work in automotive contexts. This paper presents a centralized MLOps platform designed specifically for software defined vehicles, handling the complete ML lifecycle from development to deployment, monitoring and continuous improvement. The platform uses containerization, model versioning, automated validation, edge optimized inference to manage complexity while at the same time delivering the operational excellence required by Software Defined Vehicles.
A USAF cadet and a Lincoln Laboratory researcher found AI chatbots can help nontechnical service members produce viable software applications for their unique problems.
