It is proved that sketching dimension m = O(k^{3/2}/\epsilon^2) suffices for subspace embedding with a Khatri-Rao sketching matrix with any fixed order $d$.
Abstract
We study random sketching matrices with Khatri-Rao structure. In particular, we consider the Khatri-Rao product (i.e., column-wise tensor product) $A_1\odot\cdots\odot A_d \in \mathbb R^{(n_1 \cdots n_d) \times m}$ of random matrices $A_i \in \mathbb R^{n_i \times m}$ whose columns are isotropic, independent and sub-Gaussian (e.g., Gaussian matrices). Khatri-Rao sketching matrices are widely applied in randomized algorithms for linear algebraic computation and data analysis, when the input data has tensor structure that allows for fast multiplication with $A_1\odot\cdots\odot A_d$. However, existing theory is not able to fully explain their performance in practice. In particular, despite significant attention, our best bounds for the important \emph{oblivious subspace embedding} property with Khatri-Rao matrices lag behind what is achievable with standard unstructured matrices. For embedding a $k$-dimensional subspace to $(1\pm \epsilon)$ error, Bujanovi\'c et al. \cite{bujanovic2025subspace} prove that sketching dimension $m = O(k^{3/2}/\epsilon^2)$ suffices in the special case of $d = 2$. Their dependence on $k$ is weaker than the tight bound of $O(k/\epsilon^2)$ known for unstructured sub-Gaussian sketching matrices. In this work, we close this gap, showing that $m = \tilde O(k/\epsilon^2)$ suffices for subspace embedding with a Khatri-Rao sketching matrix with any fixed order $d$. Our proof is simple, leveraging just two basic properties of the Khatri-Rao sketching distribution: 1) the columns of $A_1\odot\cdots\odot A_d \in \mathbb R^{(n_1 \cdots n_d) \times m}$ are independent and isotropic, and 2) each column of $A_1\odot\cdots\odot A_d \in \mathbb R^{(n_1 \cdots n_d) \times m}$ satisfies a weak Johnson-Lindenstrauss type moment property.
We prove that fully independent SparseStack achieves the oblivious subspace embedding parameters conjectured by Nelson and Nguyen (FOCS 2013): $m=O((d+\log(1/\delta))/\varepsilon^2)$ rows and $s=O(\log(d/\delta)/\varepsilon)$ nonzero entries per column for distortion $\varepsilon$ and failure probability $\delta$ on any fixed $d$-dimensional subspace, with explicit constants. The proof turns random-matrix concentration into a problem in finite-dimensional linear algebra. A coupling first reduces the moment estimates to a model with independent finite-valued entries. We represent these variables by multiplication operators, so their matrix moments become exact matrix elements of a deterministic operator on a finite tensor product. The central estimate bounds the contribution of $\ell\ge1$ occupied sites sharing the external factor $\mathbb{R}^d$ by $d+\ell-1$ rather than $d\ell$, yielding additive dependence on the dimension and the moment order. This approach controls both spectral edges without Gaussian comparison. The main theorem has been formally verified in Lean 4.
The spectral method uses a decoupling argument recently introduced by Kaushik, Romberg, and Muthukumar to control nonlinear error terms to control nonlinear error terms.
Let $$X_n=\sum_{i=1}^s t_i A_i^{\otimes n},$$ where $A_1,\ldots,A_s\in M_d(\mathbb C)$ and $t_1,\ldots,t_s\in\mathbb C$ are fixed, while $n$ grows. Direct computation of determinants or Schatten norms of $X_n$ is exponential in $n$. For a single tensor power these quantities are elementary, and even the determinant of a generic two-term combination admits a reduction to polynomially many scalar factors; however, no analogous elementary reduction is available for three or more terms. We give an exact representation-theoretic method which, for fixed $d$ and $s$, computes $\|X_n\|_p$, $0<p<\infty$, and determinants in polynomial time in $n$. Schur--Weyl duality yields a simultaneous block decomposition, while Jacobi--Trudi identities in the Grothendieck ring replace Schur modules by signed combinations of tensor products of symmetric powers. For $d=3$, each irreducible contribution reduces to the difference of two explicitly computable symmetric-power terms, leading to an open-source implementation. In a single-thread CPU benchmark, a genuine three-term $3\times3$ trace-norm problem with $n=18$ is evaluated in about $47$ seconds, whereas just storing the unreduced matrix would require approximately $2.4\times10^{18}$ bytes. Direct and reduced computations agree to relative error below $3.4\times10^{-15}$ throughout their common range $n\leq9$.
We develop a novel approach to matrix discrepancy based on matrix small-ball estimates. Specifically, we use a determinantal weight (obtained from the log-barrier) to scale the small-ball probability into a partition function of a tilt of the Gaussian measure. We then employ matrix-weighted Poincar\'e inequalities to compare this partition function to that of a pinched or diagonal part of the matrix, obtaining \emph{dimension-free} constants. Our technique yields a hereditary small-ball estimate for Gaussian series that should be of independent interest. As the main application, we resolve the Matrix Spencer conjecture: for symmetric $n\times n$ matrices $A_1,\dots,A_n$ with $\|A_i\|\le1$, one can efficiently find a coloring $x\in\{\pm1\}^n$ with $\|\sum_{i=1}^n x_iA_i\|=O(\sqrt n)$.
Let $d_n=(n-1)^2+1$, the dimension of the real linear span of the $n\times n$ permutation matrices. We prove that $d_n$ independent uniformly random permutation matrices are linearly independent with probability $1-O(n^{-1/2})$. Conditioning on distinctness gives the same conclusion for a uniformly random $d_n$-element subset, thereby confirming a conjecture of Kushwaha and Tripathi. The proof combines three ingredients: a mod-$2$ complexity parameter for assignment functionals, the characteristic-function estimate of Roos in the form recorded by Do--Nguyen--Phan--Tran--Vu, and a kernel decomposition argument of Ferber--Kwan--Sauermann. For the uniform-subset model, we also record the elementary lower bound $\exp(3/2+o(1))n^2e^{-n}$ coming from an unoccupied matrix position.
We consider the probability that a discrete random matrix $M_n(\xi)$ is \emph{strongly non-singular}, meaning all its leading principal submatrices are non-singular. This property is equivalent to the existence of an LU factorization. We show that for any discrete random variable $\xi$ with finite support and $|\xi|_\infty<1$, there is a constant probability that $M_n(\xi)$ is strongly non-singular with a growth factor bounded by $n^{5/2+\delta}$. Furthermore, we provide a tight asymptotic lower bound for this probability as $|\xi|_\infty \to 0$. Finally, we provide exact counts for strongly non-singular binary matrices up to $n=9$ and use these to derive improved upper bounds for the Bernoulli case.
S. Mateo, John Urschel, Nicholas West· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.
