Guan-Hong Chen

Dimensional Collapse in VQVAEs: Evidence and Remedies

This work identifies a surprising yet consistent phenomenon that it is identified: despite using high-dimensional embeddings, VQVAEs tend to compress their representations into a much smaller subspace, typically only 4 to 10 dimensions, and proposes Divide-and-Conquer VQ, which partitions the latent space into multiple low-dimensional subspaces, each quantized independently.