A Product Manifold Method for Feature Selection

Abstract

Feature selection is indispensable for mitigating overfitting and reducing feature redundancy in high-dimensional scenarios. However, most existing approaches rely on unstable overall separability and sample-wise geometry, thereby neglecting the stable class-specific discriminative structures and leading to poor performance especially in small sample regimes. Although product manifolds provide a natural geometric framework to model distinctive manifolds for different classes, their potential in feature selection is largely unexplored, e.g., theoretical guarantee on spectral convergence. In this paper, we propose a novel supervised feature selection method named PRISM, which leverages product manifold to theoretically disentangle class-specific features from the shared structure. Grounded in spectral analysis on product manifolds, we explicitly model the feature space to distinguish between shared and class-specific structures. To disentangle these class-specific structures, we design a spectral filtering mechanism that suppresses the shared components and iteratively extracts class-specific latent variables. Based on these extracted variables, we establish a scoring mechanism that identifies features with both high discriminative power and strong class specificity in high-dimensional small-sample scenarios. Crucially, we bridge the theoretical gap in spectral analysis and provide a theoretical guarantee for our method by deriving an asymptotic convergence proof under the product manifold setting, guaranteeing the reliable isolation of class-specific discriminative structures. Comprehensive experiments demonstrate that PRISM not only improves generalization performance and robustness to small sample size over leading baselines, but also achieves superior result reusability when new classes emerge.