Comparative Evaluation of Synthetic Data Generation Methods for Binary Classification Problems with Class Imbalance

Abstract

Class imbalance is a critical challenge in the classification of tabular data, since it affects the diagnostic capacity of models in domains such as health and finance. This research compares four synthetic data generation paradigms: traditional interpolation (SMOTE-NC), deep generative models (CTGAN and TVAE), and a hybrid scheme (SMOTE-NC-CTGAN) in which SMOTE-NC only densifies the minority data used to train the generator. Five UC Irvine datasets with imbalance ratios between 2.55:1 and 29.99:1 were used under a dual evaluation that combines multidimensional statistical fidelity (Wasserstein distance, correlation and categorical differences, and diversity) and predictive utility (AUC-PR, Recall, Precision, F1-Score, Specificity and G-mean), with a cost-sensitive baseline as reference. Each experiment was repeated in 10 runs that regenerate the synthetic data, and the differences were tested with paired tests, Holm correction, and effect sizes. SMOTE-NC obtains the highest fidelity in all datasets, while the CTGAN-based methods generate the most dispersed records under extreme scarcity. Synthesis does not improve the global ordering of predictions: of 40 comparisons, AUC-PR decreases significantly in 19 and increases only in 2, both in the most imbalanced dataset with XGBoost (HYBRID: 0.5252 versus 0.4872), an effect not observed with CatBoost. In contrast, Recall increases significantly in 27 of the 40 comparisons (e.g., from 0.1617 to 0.5835 with CTGAN in DS1), at the cost of Precision, and a class-weighted baseline produces a similar displacement. Data synthesis mainly shifts the decision boundary towards the minority class, which is favorable where false negatives dominate the cost.