Optimizing model-based generated tests for safety-critical embedded software
TL;DR
A test suite optimization approach that leverages symbolic classification, a greedy algorithm, and a similarity measure reduces MBT-generated test suites for embedded software by identifying and eliminating redundancy while minimizing its impact on the fault detection rate.
Abstract
Abstract The test suites generated through model-based testing (MBT) often include redundant test cases that neither enhance coverage nor fault detection but instead reduce test execution efficiency. Such test cases should be discarded to minimize the test suite size and its effect on execution cost while preserving fault detection capabilities. In this paper, we present a test suite optimization approach, implemented in a tool, that leverages symbolic classification, a greedy algorithm, and a similarity measure. It reduces MBT-generated test suites for embedded software by identifying and eliminating redundancy while minimizing its impact on the fault detection rate. We comparatively evaluate the optimized test suites against the original MBT-generated and also the manually created test suites, focusing on fault detection effectiveness and test execution efficiency. We also examine the robustness of proposed approach using two substantially different industrial case studies from Alstom Rail AB, Sweden. Results showed a significant reduction of over 80% in test suite size with a minimal effect on fault detection effectiveness. We further find that the test execution time of optimized test suites is equivalent to that of manually created ones, while achieving a fault detection rate ranging from 87% to 100%. Hence, results indicate the effectiveness and robustness of the proposed approach despite different redundancy sources (i.e., behavioral and structural) across the considered systems during the MBT test generation process.
