2023· International Journal of Machine Learning and Predictive Analytics· Vol 6, pp. 01-14· 0 citations
TL;DR
The comparative analysis indicates that there is no universal algorithm that has the best performance in all high-dimensional conditions; the performance highly depends on the sample size, the features correlation, the level of data distribution as well as noise.
Abstract
High-dimensional data are now ubiquitous in the modern science and industry, such as bioinformatics, text mining, computer vision, finance, and cybersecurity. A prominent feature of such data is having many features in comparison with the number of observations, which may cause the judgement problem of the curse of dimensionality, greater computational cost, feature overlap, and overfitting. Though supervised learning algorithms are extensively used to do predictive modeling, they have very different performance properties in high dimensional feature space. The paper contains a thorough comparison of some of the most popular supervised learning algorithms in the high-dimensional data analysis scenario. The paper provides a systematic comparison between the linear, non-linear, probabilistic, and ensemble-based classifiers, which are: Logistic Regression, Support Vector Machine, k -Nearest Neighbor, Decision Tree, Random Forest, Naive Bayes, and Artificial Neural Network. Special attention is given to the study of the behaviour of an algorithm based on scalability, ability to generalize, resistance to noise, feature sparsity, and interpretability. Besides, the paper explores how dimensionality reduction and feature selection methods impact on the performance of classification. It suggests a single experimental procedure with standardized preprocessing pipelines, cross-validation schemes and performance metrics accuracy, precision, recall, F1-score and cost of the computation. To give the concept theoretical background, mathematical formulations of learning objectives and decision functions are given. The comparative analysis indicates that there is no universal algorithm that has the best performance in all high-dimensional conditions; the performance highly depends on the sample size, the features correlation, the level of data distribution as well as noise. This study has practical implications on researchers and practitioners to consider the proper supervised learning model to use the high-dimensional datasets and identifies future research opportunities in scalable and interpretable learning.
The processing of big data has become a top priority, especially with the emergence of machine learning algorithms. Standard models suffer from several limitations and lack robustness. This paper studies four machine learning models: SVM, XGBoost, Random Forest, and logistic regression, applied to the Titanic dataset for binary classification. Model performance is evaluated using the following metrics: precision, recall, accuracy, F1-score, and area under the ROC curve (AUC), as well as through confusion matrix analysis. We applied a 5-fold cross-validation method, and statistical significance was assessed using the Wilcoxon test. Based on the analysis of the results, logistic regression achieved the best performance on the test set in terms of precision and AUC of the ROC curve, while Random Forest and XGBoost provided competitive and stable results across the validation folds. SVM also performed well, but it was slightly less effective. The Wilcoxon test shows no statistically significant difference between the models (p > 0.05). Examination of the confusion matrix reveals a tendency for ensemble methods to minimize false negatives. Overall, logistic regression demonstrates strong and good generalization ability, while ensemble methods constitute robust alternatives for binary classification on structured data.
Zahra Benider, H. Bouzahir, Jaafar Idrais· EPJ Web of Conferences· 0 citations
High-dimensional data analysis has become increasingly important in machine learning, computational intelligence, bioinformatics, finance, and image processing due to the rapid growth of large-scale datasets generated by IoT, cloud computing, and digital technologies. These datasets often contain thousands of features with limited samples, creating challenges such as overfitting, high computational complexity, redundancy, and the curse of dimensionality. Sparse machine learning models address these issues by using sparsity constraints and regularization techniques to select the most relevant features while eliminating irrelevant data. Popular methods include LASSO, Elastic Net, Sparse PCA, Sparse Autoencoders, and Sparse Support Vector Machines. These approaches improve interpretability, scalability, computational efficiency, and predictive performance. Sparse learning is widely applied in genomics, cybersecurity, medical diagnosis, recommender systems, and industrial automation. This study reviews sparse optimization, feature selection, and sparse representation learning techniques for high-dimensional data analysis. Experimental findings show that sparse models provide better robustness, faster convergence, lower memory usage, and reduced training complexity than traditional dense models. Future research focuses on integrating sparse learning with deep learning, federated learning, reinforcement learning, and explainable AI systems.
Rajesh K. Sharma, Priya Natarajan· International Journal of Mac...· 0 citations
The feature selection is a crucial step in predictive analytics to determine which subset of features makes the most contribution to the high-dimensional data and remove irrelevant, redundant, or noisy features. The dimensionality of datasets keeps on growing, and, as contemporary data-driven applications produce large volumes of heterogeneous data, overfitting, computational complexity, worse model interpretability, and poorer generalization become issues as heterogeneous data increases. The feature selection methods are meant to address such challenges by improving predictive accuracy, minimizing training time and improving model robustness. This survey is a systematic and extensive overview of feature selection methods used in predictive analytics which are utilized in a variety of areas and fields, including healthcare, finance, bioinformatics, cybersecurity, and smart systems. In the paper, the features selection techniques have been classified as filter, wrapper, embedded, and hybrid techniques which give a comprehensive theoretical background of each of the techniques as well as a comparison of each of the techniques. Statistical, information-theoretic, similarity-based, and probabilistic filters are discussed in addition to the heuristic and metaheuristic wrapper methods, i.e. evolutionary, swarm-based etc. Also critically analyzed is embedded techniques that make use of regularization, decision trees, and ensemble learning. Moreover, this survey talks about the evaluation metrics, benchmark data, and design considerations of the experiment which are used in the evaluation of the effectiveness of the feature selection. Such practice issues as scalability, stability, data imbalance, and interpretability are mentioned, as well as new directions related to deep learning-based feature selection and multi-objective optimization and explainable artificial intelligence. This piece of work can be regarded as a useful source of information by the researcher and practitioners who want to develop effective, precise, and understandable predictive analytics systems.
Nesca Mthethwa, Thane Nkosi· International Journal of Mac...· 0 citations
Credit card fraud detection is a challenging task due to the increase in digital transactions, evolving fraud patterns, and severe class imbalance present in real-world datasets. Although several comparative studies have been done in this domain, many of them are limited to a single dataset and primarily focus on predictive performance. This study presents a comparative evaluation of Supervised and Unsupervised Machine Learning algorithms for credit card fraud detection across five diverse Kaggle datasets with varied sizes, class distributions, and feature representations, including PCA-transformed numerical datasets and real-world-like datasets having numerical and categorical features. Eleven Supervised Machine Learning algorithms, namely Logistic Regression, Support Vector Machine, Decision Tree, Random Forest, Gradient Boosting, Naïve Bayes, K-Nearest Neighbors, LightGBM, XGBoost, CatBoost, and AdaBoost, together with four Unsupervised Machine Learning algorithms, namely K-Means, DBSCAN, Isolation Forest, and One-Class SVM, were evaluated using a consistent experimental framework. To keep alignment with real-world transaction environments, no sampling, resampling, or class-balancing techniques were applied. Performance was assessed using Accuracy, Precision, Recall, F1-score, ROC AUC, PR AUC, MCC, learning time, and execution time. Experimental results show that supervised ensemble methods consistently outperformed unsupervised approaches across all datasets. XGBoost and CatBoost achieved consistently strong fraud detection performance in terms of ROC AUC, PR AUC, and MCC across diverse datasets while keeping reasonable computational efficiency. Their ability to achieve high fraud detection performance under severe class imbalance makes them suitable for practical fraud detection systems. The findings provide practical insights into the robustness, scalability, and suitability of machine learning algorithms for credit card fraud detection under varying real-world data conditions that helps researchers and scientist for its better understandings.
N. Mishra, Rakhi Garg· 2026 7th International Confe...· 0 citations
The phenomenal growth in the use of data-intensive applications in areas including bioinformatics, computer vision, cybersecurity, finance, and natural language processing has resulted in the recent essential expansion of high-dimensional data sets with vast amounts of features, variables, or attributes. Although such datasets offer greater representational power and enhanced modeling expressiveness, they also introduce significant computational, statistical, and algorithmic complexity. Classical machine learning models developed for moderate-dimensional data often experience degradation in performance, scalability, and generalization in high-dimensional spaces due to the curse of dimensionality, leading to increased computational cost, overfitting, sparsity challenges, and reduced interpretability. To address these issues, scalable machine learning has emerged as a critical research area focusing on algorithmic efficiency, distributed learning, dimensionality reduction, and regularization strategies. Modern scalable approaches integrate optimization theory, parallel computing, and representation learning to efficiently process large high-dimensional datasets. Techniques such as sparse learning, ensemble-based dimensional decomposition, kernel approximation, and deep representation learning provide a balance between scalability and predictive accuracy. This paper presents a systematic analysis of scalable machine learning models for high-dimensional data, outlining structural challenges, reviewing scalable learning paradigms, and proposing a unified methodological framework that integrates feature reduction, model parallelism, and adaptive optimization. Using multiple benchmark datasets, we evaluate trade-offs among accuracy, computational efficiency, and scalability. Experimental results show that hybrid frameworks combining dimensionality reduction with distributed learning outperform standalone methods in both predictive performance and runtime efficiency. The paper contributes (i) a hierarchical taxonomy of scalable learning strategies for high-dimensional data, (ii) a modular methodological framework for scalable deployment, and (iii) an empirical evaluation supporting practical adoption by researchers and practitioners.
L. O'Connor· International Journal of App...· 0 citations
The problem of class imbalance in machine learning classification is widely present and difficult across the machine learning area, especially in real-world tasks, including fraud detection, medical diagnosing, network intrusion detection and fault prediction. When this occurs, the minority population is more likely to capture the important occurrences and the conventional machine learning models normally focus on the majority population and give misleading accuracy with poor generalization and high costs of misclassification. This paper is the result of an extensive research into powerful machine learning techniques in the classification of imbalanced datasets. The paper presents a systematic review of theoretical underpinnings of learning imbalance, literature reviews on state-of-the-art methods, such as data, algorithm-level and ensemble based methods, and suggests a convergent system methodology to build a robust classifier. Linear resampling algorithms, cost-effective learning algorithms, hybrid ensemble algorithms, and imbalanced data evaluation metrics are discussed in details. An organized experimental procedure is described to measure robustness when imbalance ratios and various noise levels are changing. Comparative findings indicate that hybrid methods that combine adaptive resampling and cost sensitive loss functions are always better than simpler classifiers based on their F1-score, G-mean, and area under the precision-recall curve. The discussion demonstrates practical trade-offs between model performance, model complexity and interpretability. In the conclusion part, the paper highlights future research directions which include scalable imbalance learning, deep learning adaptations and domain aware evaluation strategies. The paper is an excellent source of information to a researcher and practitioner aiming at finding principled and effective solutions to imbalanced classification problems.
Vasanth Kumar, Girija Rajan· International Journal of Mac...· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.
