Aug 2026· Proceedings of the 32nd ACM SIGKDD Conference on Knowledge Discovery and Data Mining V.2· pp. 2319-2329· 0 citations· 20 references
Abstract
Domain generalization (DG) aims to learn predictive models from multiple source domains that maintain performance on unseen target domains. Distributionally robust optimization (DRO) addresses distribution shift by minimizing the worst-case risk over an uncertainty set of plausible test distributions. However, if this uncertainty set is overly large, it may include unrealistic shifts, leading to low-confidence predictions. To address this issue, we propose Causal Structure-guided DRO (CS-DRO), which estimates a directed acyclic graph (DAG) that encodes the predictive relationships between representations and labels, serving as a proxy for causal structure shared across source domains. Using gradient-based signals, we quantify how well the candidate distributions explored during optimization maintain the estimated structure and impose a structure-preserving constraint on the uncertainty set. The resulting DRO objective is reformulated via Lagrangian relaxation into a tractable primal–dual learning problem. Experiments on standard DG benchmarks show that our method achieves competitive performance compared to state-of-the-art methods while improving robustness to unseen target domains. Our Github repository including code and supplementary material is here: https://github.com/gyeomo/CS-DRO.
As a popular optimization scheme, distributionally robust optimization (DRO) protects decisions against ambiguity in probability distributions. For (single-stage) DRO, prevailing dual reformulations can become difficult when model or ambiguity-set structures are complex. We study DRO from a primal perspective, working directly with distributions in ambiguity sets on closed, potentially unbounded sample spaces. This perspective leads to an algorithmic framework, referred to as BiCS, that constructs and leverages distribution cuts to achieve strong performance. We show that BiCS is applicable to standard DRO, almost-sure DRO, DRO with various chance constraints, and DRO with ambiguity sets strengthened by local information. Numerical experiments with moment and Wasserstein ambiguity sets show that this framework demonstrates superior performance, including solving cases where the examined compact reformulations are unavailable or computationally difficult. The local-information study also makes changes in worst-case distributions directly visible.
Experimental results demonstrate that proposed Bayesian domain weighting method could achieve stable and efficient domain weights learning, and identifies optimal mixtures while consuming substantially less data than search-based function-fitting methods, revitalizing optimization-based domain weighting for large-scale applications.
Xiang Yuan, Kai-Qing Lei, Zhenyu Jin et al.· arXiv.org· 0 citations
This study focuses on learning causal directed acyclic graphs (DAGs) under heteroscedastic noise models (HNMs), where each effect is modeled as a function of its causes and a Gaussian noise term whose variance depends on the causes. While HNMs theoretically guarantee identifiability of causal structures, we show that gradient-based continuous DAG learning can fail in practice due to an adverse interaction between heteroscedastic likelihood optimization and the acyclicity constraint. Specifically, because the reconstruction gradient is scaled by the predicted variance, it can be heavily attenuated in early training; as a result, the DAG parameters may be updated primarily by the acyclicity constraint before the data reconstruction signal is sufficiently learned, hindering effective structure learning. We identify and formalize this failure mode. To mitigate it, we propose a graduated optimization strategy based on a surrogate loss that decouples the variance term from the reconstruction loss, thereby preventing early gradient attenuation. We further introduce a scheduling coefficient that initially assigns a high weight to the surrogate loss for stable mean learning, and then gradually transitions to the full heteroscedastic likelihood to refine variance estimates and strictly enforce acyclicity. This strategy avoids the identified failure mode and guides the learned DAG to better reflect the data. Experimental results on both synthetic and real-world datasets verify the effectiveness of our approach. Our Github repository including code and supplementary material is here: https://github.com/Sinegi/HNM.
Eunjung Choi, Seonggyeom Kim, Dong-Kyu Chae· Proceedings of the 32nd ACM...· 0 citations
Machine learning models continue to face challenges in out-of-distribution (OOD) generalization, where domain generalization (DG) aims to improve performance on unseen domains under distributional shifts. A prevalent paradigm in DG focuses on learning domain-invariant feature representations. However, feature representations from existing methods often exhibit weak interpretability. To bridge this gap, we propose Sparse Additive Domain Generalization (SpADG). We incorporate an additive structure into the DG framework and employ ℓq,1 -norm regularization to induce sparsity, thereby enabling structured feature selection and enhancing interpretability. We present two distinct realizations: an additive kernel-based formulation and a neural additive model-based approach. The former leverages the representer theorem for flexible data adaptation, while the latter learns nonlinear shape functions. Theoretically, we derive generalization error bounds for both realizations and prove the feature selection consistency of our method under rate-scaled regularization condition. Empirical evaluations on synthetic and real-world datasets validate the effectiveness of SpADG, particularly its robustness in high-dimensional settings.
Jiayi Wang, Han Li· Proceedings of the 32nd ACM...· 0 citations
SVI-DAG is proposed, a structured variational inference approach to Bayesian causal discovery using observational data and prior beliefs that uses normalizing flows to model dependencies between edges, supporting expressive and multimodal posterior learning over DAGs.
Domain generalization (DG) aims to learn from multiple source domains and generalize to unseen target domains. Most DG methods pursue invariance: they seek a causal representation whose prediction rule is invariant across domains. This principle is effective when the causal mechanism is stable, but becomes restrictive when the domain itself modulates how causal content maps to the response. In this case, directly feeding domain style into the predictor can create misleading shortcuts, since style does not by itself cause the response. Yet the apparent chaos of multiple styles can become a ladder: style can locate the unseen target domain among source domains and guide which domain-dependent prediction rules should be trusted. We propose \emph{Latent Adaptive Domain Disentanglement and Environment Reweighting} (LADDER), a fixed-model DG pipeline that learns causal/style representations, freezes the encoders, fits source-specific classifiers, and uses an unlabeled target-domain covariate set only at inference to compute weights over these fixed classifiers, with no target labels or model-state updates. We establish theoretical guarantees for source reweighting and validate LADDER on simulations, FMoW, and a location-grouped iWildCam protocol, with gains in overall and group-averaged accuracy.
