This work proposes SURE-Ridge, a non-iterative, closed-form estimator for equal variance linear Gaussian SEM, which achieves the lowest structural Hamming distance in the small-sample regime and the lowest run time across all sample sizes tested, compared with NOTEARS, DAGMA, and GBNSL baselines.
Abstract
Recovering the directed acyclic graph (DAG) of a structural equation model (SEM) from observational data is a central problem in causal discovery. The iterative gradient descent and per-problem hyperparameter tuning of continuous-optimization methods are poorly suited to two practically important regimes: the sample-limited regime, where the number of samples is comparable to or smaller than the number of nodes in the DAG, and the compute-limited regime. This work proposes SURE-Ridge, a non-iterative, closed-form estimator for equal variance linear Gaussian SEM. The method performs parallel node-wise regressions with regularization parameters chosen adaptively by Stein's unbiased risk estimate (SURE), and applies an adaptive thresholding procedure to extract a DAG from the resulting soft adjacency matrix. Numerical results show that SURE-Ridge achieves the lowest structural Hamming distance in the small-sample regime and the lowest run time across all sample sizes tested, compared with NOTEARS, DAGMA, and GBNSL baselines.
We propose a unified Bayesian framework namely robust DAG-Cholesky horseshoe (R-DACH) for joint directed acyclic graph (DAG) structure learning and precision matrix estimation in the high-dimensional proportional asymptotic regime $p/n \to c \in (0,\infty)$, under the scale mixture of normal errors. The construction places a global-local horseshoe-type prior directly on the strictly lower-triangular entries of the modified Cholesky factor of the DAG-Markov precision matrix, so that sparsity in the Cholesky parameters induces a coherent parent-set selection consistent with a topological ordering of the variables. A per-observation inverse-gamma scale mixture yields automatic robustness to heavy-tailed and contaminated observations and admits Student-$t$, Laplace, and slash distributions as special cases. We design a partially-collapsed blocked Gibbs sampler that traverses the joint space of orderings, sparsity patterns and continuous parameters. Simulations across $(n,p)$ configurations with $p$ up to several hundreds confirm the theoretical rates and demonstrate substantial gains over graphical-horseshoe, DAG-Wishart, and PC-based competitors under contamination. An application to RNA-seq gene-expression data from \emph{The Cancer Genome Atlas} reveals biologically interpretable regulatory structure that competing methods fail to recover.
Samaneh Nazari, Mohammad Arashi, Abdolnasser Sadeghkhani· 0 citations
We study the squared $2$-Wasserstein distance to the standard Gaussian as a non-Gaussianity criterion and use it for linear Independent Component Analysis (ICA) and causal discovery in Linear Non-Gaussian Acyclic Models (LiNGAM). Unlike commonly used parametric contrasts and approximations of information-theoretic quantities, this criterion requires no distributional regularity beyond finite second moments, involves neither approximation nor tuning parameters, and can be computed exactly and efficiently from empirical order statistics. Our analysis relies on a strict subadditivity property of the $2$-Wasserstein distance to the Gaussian. At the population level, we prove exact identification of the ICA unmixing matrix, up to signed permutation, and give an analogous characterization of causal orders through sequential least-squares residuals. We then define empirical plug-in estimators and prove distribution-free uniform convergence under finite-moment assumptions, before detailing three practical solvers: a Picard-style orthogonal optimizer for ICA, an exhaustive dynamic program for causal order search, and a greedy order search variant. Empirically, we demonstrate competitive performance for both tasks and provide open-source implementations for source separation and causal discovery.
F'elix Laplante, C. Ambroise, Pierre Humbert· 1 citation
Recent algorithmic advances have made directed acyclic graph (DAG) structure learning scalable for causal discovery. Yet, the currently available techniques assume a completely homogeneous population, precluding their application to clustered data where cluster-specific variations (e.g., patient-specific effects) are common. We address this issue by introducing a new approach that estimates a global structure while accounting for local cluster-level effects. The key idea is to extend the fixed- and random-effects framework of classical mixed models to the structure learning setting. Towards this end, we present a differentiable graph coupling mechanism that guarantees the union of the fixed- and random-effects graphs remains acyclic. Computationally, we provide a provably convergent first-order method and leverage efficient batched updates across clusters. Statistically, we establish identifiability of the model and show that our approach recovers the true structure asymptotically. In experiments on real and synthetic data, our proposal detects dependencies missed by alternative estimators, underscoring its value for structure learning in clustered settings.
Ryan Thompson, Matt P. Wand, V. Baladandayuthapani· 0 citations
Kernel ridge regression is a standard method for functional data analysis, but its exact behavior is less understood. We study tensor-product kernel ridge regression for estimating the $r$-th moment function of a random function based on noisy discrete observations. The formulation includes mean estimation, covariance estimation, and higher-order moment estimation in a single framework. Our main result gives a precise $1+o_{\mathbb{P}}(1)$ expansion for the $L^2$ error at each admissible regularization parameter. The expansion consists of bias and three variance terms corresponding respectively to variation across the independent sample paths, latent signal variation at each sample point, and variation from measurement errors, identifying the refined error structure underlying functional data. As applications, we show that KRR attains the minimax rate for source smoothness $s \leq 2$ but becomes suboptimal in the sparse regime for $s>2$ due to saturation. A technical ingredient is a set of concentration inequalities for $U$-statistics suited to the dependent product structure of functional observations.
Broken adaptive ridge (BAR) penalty approximates $L_0$-regularization through iterative reweighting of L2 penalties. This penalty enjoys both the oracle property and the grouping effect for highly correlated covariates, making it particularly attractive for penalized regression with complex dependence among predictors. In this paper, we develop a BAR-penalized linear rank regression method for the semiparametric accelerated failure time model with right-censored data. Computational tractability is achieved by applying induced smoothing to the nonsmooth Gehan-type rank estimating function, yielding a more stable framework for estimation and inference. For scalable penalization, we develop a cyclic coordinate descent algorithm that minimizes the penalized objective function, and estimates the regression coefficients in a coordinate-wise manner. We further extend the proposed method to more complex survival endpoints, such as multivariate partly interval-censored (PIC) data. Under mild conditions, the proposed estimator satisfies both the oracle property and the grouping effect, and the variance estimator of the informative coefficients can be derived in analytic form. Numerical studies using synthetic data compare our approach to several well-known penalties, and demonstrate its superior selection accuracy and estimation efficiency across various scenarios. Furthermore, applications to right-censored outcomes from primary biliary cirrhosis, and correlated PIC outcomes from colorectal cancer further illustrate the practical utility of the proposed method. The R package aftPenCDA for implementing the method is available on R CRAN.
Su-Yeon Seon, Dipankar Bandyopadhyay, Seongoh Park et al.· 0 citations
This dissertation advances the field of robust multivariate statistics by developing new methods that are well-suited for data containing outliers and heavy tails. By introducing targeted robust procedures, including specialized estimators and tailored data transformations, we adapt four existing statistical methods and demonstrate their effectiveness through simulations and real-world applications. First, we propose a computationally efficient robust principal component analysis method (PCA) based on decomposing the generalized spatial sign covariance matrix. This approach overcomes the computational bottlenecks and sample-size limitations of existing robust PCA methods while maintaining flexibility through various radial functions. Second, in the domain of causal inference, we introduce TSLiNGAM. By replacing the non-robust ordinary least squares regression in the original LiNGAM algorithm with Theil-Sen regression, this method achieves superior causal discovery on heavy-tailed and skewed data. Third, addressing the need for a highly robust independence measure, we introduce the biloop data transformation to robustify distance correlation. The resulting biloop distance correlation achieves a bounded, redescending influence function and a non-zero breakdown point. Fourth, we leverage the biloop distance correlation to develop a new approach for independent component analysis (ICA). By applying a novel transform called bowl to the dCovICA method, we effectively separate multivariate sources in the presence of outliers. Finally, to bridge the gap between robust statistics, traditionally more established in R, and Python-based data analysis, we present RobPy, a comprehensive Python library for robust regression, PCA, covariance estimation and cellwise outlier detection.
Sarah Leyder· 0 citations
Related blog posts
MIT News · Artificial Intelligence· news.mit.eduAug 18, 2026
A new method for surgically removing training examples from a model reveals that as datasets grow, the link between what a model learns and what it produces dissolves.
