ReLU: Refined Chunk Embeddings Learning for Ultra-long SNP Genomic Prediction in Crop Breeding

Abstract

Genomic prediction can accelerate crop breeding by predicting phenotypes from genome-wide SNP sequences. In practice, this task operates in a severe p \gg n regime, hundreds of thousands to millions of SNPs but only thousands of samples, making end-to-end deep learning prone to overfitting. Genomic foundation models help by providing informative pretrained embeddings, yet their limited input length due to high memory cost makes them difficult to apply to ultra-long SNP sequences. A straightforward solution is to chunk SNP sequences and pool the embeddings, but pooled chunk embeddings often lose local SNP information within the sample and hinder modeling global dependencies across samples. To address this, we present ReLU, Refined Chunk Embeddings Learning for Ultra-long SNP modeling, a scalable framework that refines chunk embeddings progressively for genomic prediction in crop breeding. ReLU introduces three key contributions: (1) A self-supervised adapter that produces compressed refined chunk embeddings to preserve local SNP information; (2) a listwise ranking loss that optimizes refined embeddings in training and (3) a retrieval-based mechanism that refines test embeddings to enhance global cross-sample dependency modeling. Experiments on two real-world datasets, Rice3K with 440K SNPs and Maize1404 with 1M SNPs, demonstrate that ReLU outperforms baselines across multiple phenotypes.