Expanding Data-Agnostic Pivotal Instances Selection Models with Proximity Trees and Ensemble Learning

NiWo: An Augmentation Framework to Enhance ML Performance and Interpretability for Tabular Data with Class Imbalance

Various data augmentation methods have been proposed to address class imbalance in Machine Learning (ML) and Artificial Intelligence tasks across multiple data modalities. For tabular data, augmentation methods must be interpretable so that human decision-makers can audit the process (e.g., which neighborhoods are being augmented and why). This is particularly essential for data from high-stakes domains. Previous studies have integrated explanation tools to derive instance-level weights that drive the augmentation process while maintaining interpretability. However, such an integration is computationally expensive for real-world datasets with thousands of instances due to the complexity of explanation tools. This paper proposes a novel augmentation framework, NiWo, that eliminates the need for explanation tools. NiWo optimizes the weights of influential neighborhood instances within an augmentation budget, i.e., the total number of records to be generated, thus preserving computational efficiency and offering interpretability. It implements two optimization strategies and an ensemble method that activates the most appropriate strategy for a given dataset. NiWo decouples budget allocation from instance generation. The latter can be flexibly replaced to maximize improvement in model performance. This also enables augmentation to be audited and adjusted based on domain-specific requirements within a human-in-the-loop framework. Results of 62 tabular datasets and 7 models show that NiWo outperforms other augmentation methods at enhancing ML performance, especially over datasets with class imbalance and scarce instances.

DICS: Data-Informed Centroid Splitting for Decision Tree Classifiers

Decision tree-based models are widely used in machine learning due to their interpretability and strong empirical performance. However, training decision trees can be computationally expensive, particularly for large and high-dimensional datasets, largely due to the exhaustive search over candidate splits at each node. To improve computational efficiency, we propose Data-Informed Centroid Splitting (DICS), a clustering-based framework that constructs a compact and informative set of candidate splits using data-driven priors. By incorporating class-aware structure, DICS significantly reduces the split search space for classification tasks while preserving predictive performance. We further provide theoretical analysis showing that under the stated assumptions, DICS does not degrade the performance of classification trees compared to exhaustive split search. DICS can be incorporated into classification trees, random forests, and gradient-boosting models. Extensive experiments demonstrate that DICS achieves comparable accuracy while substantially reducing training time across synthetic and benchmark datasets, highlighting the benefit of integrating data-informed priors into split selection for scalable classification tree learning.

Learning the Pareto Frontier of Predictive Models under Distribution Shift

Modern machine learning pipelines increasingly rely on reusing pretrained and foundation models across downstream tasks. These pretrained models can differ not only in performance but also in how they can be used: some only provide black-box predictions, while others may permit white-box access to internal representations that can be probed or fine-tuned. When deployed to the target domain in the presence of distribution shift, no single strategy, including zero-shot application, fine-tuning, or directly training a target-specific model, is uniformly the best. In this work, we propose Frontier Learning, a framework that treats a library of candidate models spanning different training histories and access regimes as complementary sources of information rather than mutually exclusive alternatives. Frontier Learning constructs a unified target-domain feature by concatenating internal representations from white-box candidates as well as prediction outputs from black-box candidates, then fits a lightweight, regularized supervised learner on this concatenated representation using labeled target data. Because the resulting hypothesis class contains predictors obtained by zero-shot reuse, fine-tuning, and direct training as special cases, empirical risk minimization over the frontier learner is guaranteed to be no worse, on the training sample, than any individual baseline. We evaluate the framework in simulations spanning varying degrees of source-target compatibility and in two real-world distribution-shift settings: visual domain adaptation on DomainNet/VisDA and clinical mortality prediction across intensive care unit domains using MIMIC-IV-Notes. Across all settings, Frontier Learning matches or outperforms the strongest individual reuse strategy, with the largest gains arising precisely when no single baseline is reliable across the range of shift considered.

MultiPruneEnsemble: Integrating Multi-Base Decision Tree Prunings via Gradient-Based Weighting

DECODE: Decision Tree Capturing Opaque Decisions via Coverage-Driven Synthetic Sampling

In the context of smart manufacturing, Explainable AI has emerged as an essential solution to ensure trust in complex Machine Learning model decisions. However, most employed methods are limited to feature relevance scoring, lacking in providing a human-interpretable description of model behaviour. Surrogate models, however, address this gap by approximating a complex predictor through an interpretable model, with treebased surrogates striking a great balance between performance, interpretability, and deployability. In this paper, we propose DECODE, a tree-based surrogate framework combined with an Empty Space Search (ESS) synthetic sampling strategy. Unlike commonly employed synthetic sampling approaches that are either contrained to a specific data distribution or focused on local decision-boundary neighbourhoods, ESS aims to maximise data coverage across a dataset's feature range, enabling the surrogate to characterise model behaviour in low-density regions that reflect rare or abnormal conditions. Experiments on multiple low-dimensional datasets and with multiple classifiers show that an ESS-trained DECODE model substantially improves model behaviour explainability, measured through fidelity and rule overlap metrics, while maintaining satisfactory predictive performance with its teacher model. These results highlight the importance of coverage-centred synthetic training to achieve faithful surrogate explanations that go beyond high density data regimes.

A Heterogeneous Mixture of Experts Framework for Interpretable Machine Learning

Mixture-of-Experts (MoE) models provide a flexible framework for partitioning complex prediction problems into simpler local learning tasks through an input-dependent gating mechanism. Existing interpretable MoE approaches, such as Mixture of Decision Trees (MoDT), achieve transparency by employing homogeneous decision-tree experts, but this restricts the model to a single inductive bias across all regions of the feature space. We extend the MoDT framework by introducing heterogeneous expert families comprising decision trees, linear support vector machines, and quadratic discriminant analysis under a common probabilistic gating mechanism. To ensure coherent likelihood-based inference, non-probabilistic experts are calibrated to produce conditional class probabilities, allowing parameter estimation within the generalized Expectation-Maximization framework of MoDT. We further establish theoretical monotone ascent guarantees for the proposed heterogeneous gating updates, providing a justification for the optimization procedure. Experiments on a diverse collection of synthetic and real-world benchmark datasets demonstrate that the proposed framework adaptively specializes experts according to local data geometry, yielding interpretable expert assignments while achieving predictive performance competitive with homogeneous MoDT and Random Forests. The proposed approach combines interpretability, adaptive inductive bias selection, and probabilistic coherence within a unified mixture-of-experts framework.