2025
Dimensional Collapse in VQVAEs: Evidence and Remedies
This work identifies a surprising yet consistent phenomenon that it is identified: despite using high-dimensional embeddings, VQVAEs tend to compress their representations into a much smaller subspace, typically only 4 to 10 dimensions, and proposes Divide-and-Conquer VQ, which partitions the latent space into multiple low-dimensional subspaces, each quantized independently.
Jiayou Zhang, Yifan Shen, Guan-Hong Chen et al.
· Neural Information Processin... · 4 citations