Applying non-negative matrix factorization with covariates to label matrix for classification
TL;DR
NMF-LAB provides a unified, probabilistic, and scalable framework for classification based on nonnegative matrix factorization, which gives rise to two complementary designs that emphasize feature-level interpretability and competitive predictive accuracy.
Abstract
Non-negative matrix factorization (NMF) is widely used for representation learning and interpretable analysis in high-dimensional nonnegative data, but standard formulations are unsupervised and do not directly address classification. Existing supervised extensions typically incorporate labels only through penalties or graph constraints and often require an external classifier. We propose NMF-LAB (Non-negative Matrix Factorization for Label Matrix), a classification framework that directly factorizes the label matrix Y while treating covariates A as explanatory variables. By modeling class labels as the target of factorization, NMF-LAB yields a direct probabilistic mapping from covariates to labels without an additional classifier. The proposed framework gives rise to two complementary designs. NMF-LAB Direct emphasizes feature-level interpretability through its nonnegative and additive structure, whereas NMF-LAB Kernel incorporates nonlinear covariates to improve predictive performance and generalization. Kernel-based similarities are integrated via Gaussian-kernel covariates, and the kernel design scales to large datasets through Nyström approximation. Extensive experiments on diverse datasets, ranging from small- and medium-scale benchmarks to the large-scale MNIST dataset, demonstrate the interpretability of the Direct design through sparse, nonnegative coefficient estimation, and the competitive predictive accuracy of the Kernel design. Overall, NMF-LAB provides a unified, probabilistic, and scalable framework for classification based on nonnegative matrix factorization.