TESTING ARTIFICIAL INTELLIGENCE COMPONENTS IN THE SYSTEM UNDER TEST
TL;DR
The paradigm shift from testing for correct instruction execution to testing for robust behavior under open‑world conditions is analyzed and practical recommendations for integrating metamorphic testing and formal robustness verification into existing validation pipelines for mPNT systems are concluded.
Abstract
The integration of AI‑based components (deep learning odometry, AI‑driven fault detection, end‑to‑end localization) into multi‑source Positioning, Navigation and Timing (mPNT) systems fundamentally changes the nature of testing. Unlike traditional deterministic algorithms, neural networks operate as “black boxes” trained on finite datasets and can fail unpredictably on unseen inputs. This paper analyzes the paradigm shift from testing for correct instruction execution to testing for robust behavior under open‑world conditions. Three core challenges are examined: the oracle problem, out‑of‑distribution vulnerability, and the risk of “hallucinations”. A comparative analysis of testing methods for classical and AI‑based navigation components is provided, followed by a detailed case study of the OdoTest framework for deep odometry. The paper concludes with practical recommendations for integrating metamorphic testing and formal robustness verification into existing validation pipelines for mPNT systems.