Sparse Additive Models for Domain Generalization
TL;DR
This work incorporates an additive structure into the DG framework and employs ℓq,1 -norm regularization to induce sparsity, thereby enabling structured feature selection and enhancing interpretability and presents two distinct realizations: an additive kernel-based formulation and a neural additive model-based approach.
Abstract
Machine learning models continue to face challenges in out-of-distribution (OOD) generalization, where domain generalization (DG) aims to improve performance on unseen domains under distributional shifts. A prevalent paradigm in DG focuses on learning domain-invariant feature representations. However, feature representations from existing methods often exhibit weak interpretability. To bridge this gap, we propose Sparse Additive Domain Generalization (SpADG). We incorporate an additive structure into the DG framework and employ ℓq,1 -norm regularization to induce sparsity, thereby enabling structured feature selection and enhancing interpretability. We present two distinct realizations: an additive kernel-based formulation and a neural additive model-based approach. The former leverages the representer theorem for flexible data adaptation, while the latter learns nonlinear shape functions. Theoretically, we derive generalization error bounds for both realizations and prove the feature selection consistency of our method under rate-scaled regularization condition. Empirical evaluations on synthetic and real-world datasets validate the effectiveness of SpADG, particularly its robustness in high-dimensional settings.