MOTIVATION
Gene expression prediction benefits from integrating DNA sequences and epigenomic signals. Existing approaches typically combine these modalities using simple operations such as concatenation or summation, without explicitly modeling fine-grained token-level cross-modal interactions. Establishing precise cro...
Li Liu, Guipeng Xv, Shu-Jie Liu et al.· Bioinformatics· 0 citations
ESRL is introduced, an architecture-aware framework that explicitly explores the expert-routing space of MoE models, and preserves high-confidence experts as anchors, and restricts stochastic routing to a plausible candidate pool, thereby retaining reliable computation paths.
Hong-Yi He, Zheng-Wen Lin, Xiao Liu et al.· 0 citations
This work proposes MSign, a new optimizer that periodically applies matrix sign operations to restore stable rank in a 5M-parameter NanoGPT model, and proves theoretically that these two conditions jointly cause exponential gradient norm growth with network depth.
Lianhai Ren, Yucheng Ding, Xiao Liu et al.· arXiv.org· 1 citation
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.