Skip to content

Author

Huai-Jin Wu

We have 2 of 15 papers

We haven’t gathered this author’s papers yet. Follow them and we’ll fetch their work.

Not the right person? Other researchers publish under this name.

#computer vision Preprint Oct 2026

Multi-Stream Attention Residuals

Attention Residuals (AttnRes) enables content-dependent retrieval over network depth, which can be viewed as single-stream softmax attention along the depth axis. Inspired by the success of multi-head attention along the sequence axis, we introduce Multi-Stream Attention Residuals (MSAR), an extension of AttnRes that g...

Xiao-Han Qin, Xiang-Dong Zhang, Tuo Dai et al. · 4 citations
#artificial intelligence Preprint Sep 2026

How Does Local Landscape Geometry Evolve in Language Model Pre-Training?

The scale and expense of pre-training language models make efficient hyperparameter tuning essential, yet a principled guidance is still missing. In this work, we analyze language model pre-training dynamics from a local landscape geometry perspective. Our study reveals two distinct phases. In Phase I, sharpness of the...

Zhan-Peng Zhou, Yu-Han Sun, Bing-Rui Li et al. · 1 citation

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.