Progressive Multi-Objective Optimization for Improved t-SNE Embeddings

Abstract

: Dimensionality reduction is essential for analyzing and visualizing high-dimensional data, with t-distributed Stochastic Neighbor Embedding (t-SNE) being widely used due to its ability to preserve local neighborhood structures. However, its reliance on a single Kullback–Leibler (KL) divergence objective often leads to poor global structure preservation and sensitivity to local inconsistencies. In this paper, we propose a progressive multi-objective optimization framework that enhances t-SNE by integrating complementary loss functions, including a ranking-aware divergence (KLmax) and a Wasserstein-based term for global alignment. Rather than optimizing all objectives simultaneously, we introduce a progressive training strategy that gradually incorporates these components, enabling more stable convergence and improved embedding quality. Additionally, the framework is applied to latent representations learned via a neural encoder, providing a more structured feature space for dimensionality reduction. Experiments on the MNIST, Fashion-MNIST, CIFAR-10, and STL-10 datasets demonstrate that the proposed method improves clustering performance and yields more interpretable embeddings than standard and extended t-SNE approaches.