Comparative Evaluation of Machine Learning Models for Predictive Maintenance Using the AI4I 2020 Dataset: Effects of Data Balancing and Feature Selection

Abstract

This article presents a comparative analysis of machine learning (ML) models applied to machine fault classification, using the AI4I 2020 dataset. The performance of five algorithms (random forest, decision tree (J48), artificial neural networks (ANN), k-nearest neighbors (KNN), and Naive Bayes) is evaluated in six different data scenarios: synthetic minority oversampling (SMOTE) technique, undersampling, and original imbalanced distribution in the complete feature set and in the feature selection set. In addition, the impact of feature selection techniques on predictive power is evaluated. The goal is to identify the optimal predictive model and compare the performance of these approaches to determine the advantages of feature reduction and data balancing strategies. To ensure robust validation, the models are analyzed using precision, recall, specificity, accuracy, and F1 score. The main conclusions obtained demonstrate that it is possible to build reliable models capable of maximizing fault detection and minimizing false alarms, which is vital for predictive maintenance systems in industries where accuracy is critical.