Author

Le Song

h-index 6 View on Semantic Scholar

We have 2 of 15 papers

We haven’t gathered this author’s papers yet. Follow them and we’ll fetch their work.

Not the right person? Other researchers publish under this name.

2025

Dimensional Collapse in VQVAEs: Evidence and Remedies

This work identifies a surprising yet consistent phenomenon that it is identified: despite using high-dimensional embeddings, VQVAEs tend to compress their representations into a much smaller subspace, typically only 4 to 10 dimensions, and proposes Divide-and-Conquer VQ, which partitions the latent space into multiple...

Jiayou Zhang, Yifan Shen, Guan-Hong Chen et al. · Neural Information Processin... · 4 citations

Open access Aug 2026

Pretraining Enhances Megabase-Scale Gene Expression Prediction with GeneUnet

GB.GeneUnet, an 837M-parameter transformer-based U-Net pretrained on 6 trillion tokens from multi-species genomes in OpenGenome2 is introduced, extending genomic context to 1 Mb with up to 100× inference speedup over GeneMoE, a preliminary MoE transformer baseline of similar model size pretrained on the same data.

Ning Sun, William de Vazelhes, Pan Li et al. · bioRxiv · 0 citations