Class Cardinality as a Source of Prediction Uncertainty in E-Commerce Customer Analytics

Abstract

Class imbalance and class cardinality both affect multiclass classification, but their influence on probabilistic estimation has been less explored. This study examines these impacts using an e-commerce dataset. The study follows a four-stage methodology comprising classifier comparative evaluation, controlled class-cardinality analysis, validation using real categorical variables, and class-imbalance evaluation. Multiclass classification tasks are evaluated using Support Vector Machine, Gaussian Naive Bayes, Logistic Regression, Random Forest, and Decision Tree classifiers. Under five-fold cross-validation, performance is assessed using the macro-F1 score, log loss, and accuracy. Results show that macro-F1 score and accuracy decrease as class cardinality increases, causing greater classification difficulty. Tree-based models like Random Forest exhibit more balanced performance across classes. Gaussian Naive Bayes obtains the lowest log loss, indicating more accurate probability estimations. Class cardinality effects are isolated by varying the number of classes while keeping the features and classifier fixed. Increasing class cardinality reduced posterior confidence and increased entropy and log loss. Using real-time categorical variables, these trends are confirmed. Class imbalance primarily affects minority class performance, whereas class cardinality exerts a broader influence on probabilistic confidence and prediction uncertainty. The findings emphasize the necessity to consider class cardinality, class imbalance, and probabilistic metrics when evaluating multiclass classification models.