Author

Shu Zhong

View on Semantic Scholar

2 papers indexed here

We haven’t gathered this author’s papers yet. Follow them and we’ll fetch their work.

Not the right person? Other researchers publish under this name.

#artificial intelligence May 2026

Negligible in Size, Significant in Effect: On Scale Vectors in Large Language Models

This work presents a systematic study of scale vectors in LLMs from the perspectives of expressivity, optimization, and architectural structure, and proposes three lightweight and complementary improvements to scale vectors: branch-specific heterogeneity, improved placement around linear mappings, and magnitude-direction reparameterization.

Mingze Wang, Shuchen Zhu, Yuxin Fang et al. · arXiv.org · 3 citations

#artificial intelligence May 2026

More Expressive Feedforward Layers: Part I. Token-Adaptive Mixing of Activations

This work proposes Mixture of Activations (MoA), a token-adaptive FFN design that mixes a dictionary of activation functions using lightweight input-dependent gates while sharing the same linear projections, suggesting that token-adaptive activation mixing is a simple and effective mechanism for improving FFN expressivity in LLMs.

Mingze Wang, Jinbo Wang, Yikuan Xia et al. · arXiv.org · 3 citations