A Structure-Revealing Tensor Network Paradigm and Its Applications.
TL;DR
This work proposes a novel structure-revealing TN paradigm, termed SVD-inspired TN decomposition (SVDinsTN), which enables the formulation of an energy function that simultaneously considers representation error and TN structure complexity, facilitating the joint optimization of data representation and the identification of a compact TN structure.
Abstract
Tensor network (TN) decomposition is a powerful data representation tool widely used in machine learning and computer vision. However, existing TN decomposition methods are limited by fixed topologies, which constrain their representational capacity. To overcome this limitation, we introduce, for the first time, an energy-based framework that allows us to reveal a data-adaptive TN structure through the optimization of an energy function. Specifically, we propose a novel structure-revealing TN paradigm, termed SVD-inspired TN decomposition (SVDinsTN). This paradigm enables the formulation of an energy function that simultaneously considers representation error and TN structure complexity, facilitating the joint optimization of data representation and the identification of a compact TN structure. Theoretically, we present the principles behind the energy function and derive an upper bound for the TN rank. In contrast to heuristic TN structure search methods, our energy-based method achieves a remarkable acceleration of 100 to 1000 times, making it practical for real-world applications. Experimental results in neural network compression, remote sensing image cloud removal, and tensor robust principal component analysis consistently show that SVDinsTN can identify a TN structure that better aligns with the given data, achieving superior performance over fixed-topology methods.