Jun 2026· arXiv.org· Vol abs/2606.31061· 0 citations· 18 references
MathematicsComputer Science
TL;DR
Online TT-ALS (Alternating Least Squares) is introduced, an algorithm that sequentially enforces orthogonality constraints that allows for efficient and exact updates of the core tensor while maintaining high reconstruction accuracy and is suitable for low-latency real-time processing applications.
Abstract
Tensor Train (TT) decomposition is a powerful technique for analyzing high-dimensional data. Existing algorithms for computing TT decompositions can be categorized into two main types: conventional batch-based approaches and recursive online methods. In the context of streaming data, batch methods typically achieve higher reconstruction accuracy but often suffer from memory exhaustion, while online methods provide greater computational efficiency. In this work, we introduce Online TT-ALS (Alternating Least Squares), an algorithm that sequentially enforces orthogonality constraints. This approach allows for efficient and exact updates of the core tensor while maintaining high reconstruction accuracy. Theoretically, we prove that enforcing these orthogonal gauge constraints guarantees monotonic decrease of the local objective function and temporal smoothness. Computationally, our deterministic single-sweep update reduces the rank dependence from quadratic to linear, achieving an overall complexity of $\mathcal{O}(I^{n-1} r)$. Experimental results demonstrate that the proposed method outperforms existing online techniques not only in terms of mathematical approximation accuracy but also in human perception-based video quality metrics. Furthermore, compared to recent deep learning-based paradigms, our algebraic approach achieves speedups of several orders of magnitude. Consequently, our method exhibits high computational efficiency and is suitable for low-latency real-time processing applications.
In this work, we present the tree tensor network Nyström (TTNN), an algorithm that extends recent research on streamable tensor approximation, such as for Tucker and tensor‐train formats, to the more general tree tensor network format, enabling a unified treatment of various existing methods. Our method retains the key features of the generalized Nyström approximation for matrices, that is, randomized, single‐pass, streamable, and cost‐effective. Additionally, the structure of the sketches allows for parallel implementation. We provide a deterministic error bound for the algorithm and, in the specific case of Gaussian dimension reduction maps, also a probabilistic one. We also introduce a sequential variant of the algorithm, referred to as sequential tree tensor network Nyström (STTNN), which offers better performance for dense tensors. Furthermore, both algorithms are well‐suited for the recompression or rounding of tensors in the tree tensor network format. Numerical experiments highlight the efficiency and effectiveness of the proposed methods.
Alberto Bucci, Gianfranco Verzella· Numerical Linear Algebra wit...· 3 citations
Tensor networks are powerful formats for compressing large-scale data. However, their application to general data processing has been limited by the difficulty of performing nonlinear operations. Here, we introduce iterative tensor network transformations (ITNTs), a general algorithmic framework for the element-wise evaluation of elementary and nonlinear filtering functions on data encoded as tensor trains (TTs), a class of tensor networks. Our approach operates entirely in the compressed domain, enabling efficient computation on exponentially large datasets while maintaining a controlled computational cost. We demonstrate its power in two key areas: (I) evaluating highly nonlinear elementary and filtering functions on a 3D reactive flow field, enabling high-fidelity reaction rate computation and region filtering, and (II) finding extrema in complex optimization problems, such as solving Max-SAT instances on spaces up to $2^{70}$ configurations. These results establish ITNT as a foundational tool that provides tensor network methods with the capability for general-purpose data science and large-scale optimization.
Xiao Wang, Tomohiro Hashizume, Pia Siegl et al.· 2 citations
We address the problem of recovering highly oscillatory operators, represented as n×n$$ n\times n $$ matrices with a fixed set of observed entries. Given that these matrices can be well compressed by butterfly matrix decomposition of L=𝒪(logn) levels requiring only O(nlogn)$$ O\left(n\log n\right) $$ degrees of freedom, we propose a novel reformulation of the butterfly structure as a compact tensor network. Specifically, we reshape the input matrix as an order 2L+2$$ 2L+2 $$ dense tensor, and cast its butterfly decomposition as a tensor network consisting of order L+3$$ L+3 $$ dense tensors. This enables efficient utilization of the existing software infrastructure for dense and sparse tensor computations. Next, we propose several tensor completion algorithms based on the tensor reformulation of butterfly format, and compare them against algorithms using the quantized tensor train (QTT) format. These algorithms leverage popular completion methods such as alternating least squares, gradient‐based optimization, and alternating directions fitting. Moreover, we propose a novel strategy that uses low‐rank matrix completion to efficiently generate an initial guess for the proposed algorithms. To demonstrate the efficiency and applicability of our proposed algorithms, we perform three numerical experiments using simulated oscillatory operators in seismic applications. In these experiments, we use 𝒪(nlogn) observed entries in the input matrix and demonstrate an 𝒪(nlog3n) computational cost of the proposed algorithms, leading to speedups of orders of magnitude and significant accuracy improvements for large matrices compared to state‐of‐the‐art low‐rank matrix and QTT tensor completion algorithms.
Navjot Singh, Edgar Solomonik, X. Li et al.· Numerical Linear Algebra wit...· 0 citations
Traditional model reduction techniques for neural network compression usually leverage existing matrix/tensor decomposition methods to explore the redundancy among model parameters, which may overlook the local structures of the models. In this work, we propose a novel low-rank approximation algorithm, called Dimension-Augmented Tensor-Train (DATT) decomposition, which integrates dimension slicing with the Tensor-Train decomposition. DATT allows different partitions to be represented using distinct, non-shared spanning sets. This enables the decomposition to adapt more flexibly to localized structure, potentially achieving better compression. With the localized low-rank structures, DATT provides a more compact representation of neural network weights. We evaluated the performance of DATT with the other state-of-the-art low-rank approximation-based model compression algorithms, and demonstrated that DATT outperforms them in terms of parameter reduction and model accuracy. In particular, for compressing ResNet-20 on CIFAR-10, DATT achieves a 77.00% reduction in parameters with only a 0.37% drop in top-1 accuracy.
Yu-Chuan Tai, Cheng-Yu Sie, Che-Rung Lee· Annual International Compute...· 0 citations
Abstract.
This work proposes an efficient numerical approach for compressing a high-dimensional discrete distribution function into a non-negative tensor train (NTT) format. The two settings we consider are variational inference and density estimation, whereby one has access to either the unnormalized analytic formula of the distribution or the samples generated from the distribution. In particular, the compression is done through a two-stage approach. In the first stage, we use existing subroutines to encode the distribution function in a tensor train format. In the second stage, we use an NTT ansatz to fit the obtained tensor train. For the NTT fitting procedure, we use a log barrier term to ensure the positivity of each tensor component and then utilize a second-order alternating minimization scheme to accelerate convergence. In practice, we observe that the proposed NTT fitting procedure exhibits drastically faster convergence than an alternative multiplicative update method that has been previously proposed. Through challenging numerical experiments, we show that our approach can accurately compress target distribution functions.
Reproducibility of computational results. This paper has been awarded the “SIAM Reproducibility Badge: Code and Data Available” as a recognition that the authors have followed reproducibility principles valued by SISC and the scientific computing community. Code and data that allow readers to reproduce the results in this paper are available at https://github.com/Xun-Tang123/Non_Negative_Tensor_Train and in the supplementary materials ( Non_Negative_Tensor_Train-main.zip [51.4MB]), linked from the main article webpage. [Formula: see text]
Xun Tang, R. Dwaraknath, Lexing Ying· SIAM Journal on Scientific C...· 0 citations
Tensor decomposition has become an important technology for recovering missing information in high-dimensional image and video data. Alternatively, spatial regularization has been incorporated into tensor models such as Tucker, tensor train (TT), and tensor ring (TR). Existing alternating least squares (ALS) algorithms are efficient but generally lack regularization. Although spatial regularization techniques have been successfully integrated into ALS-based TT, extending this strategy to TR remains unsolved because of the circular structure and the additional trace operation. This paper proposes a tensor ring alternating least squares with Tikhonov regularization subproblem (TRATR), a framework that incorporates Tikhonov regularization into the core-wise update of the TR-ALS. Extensive experiments on images and videos under different missing rates demonstrate consistent improvements in relative standard error (RSE), peak signal-to-noise ratio (PSNR), and structural similarity index (SSIM), while reducing the required rank to 63.6% of the rank and yielding up to 83.6% reduction in update complexity. These results highlight the potential of TRATR for large-scale and resource-constrained applications.