Automating AI research creates a positive feedback loop between software capability and research effort. We study a stochastic realisation of this loop in which independent, positive, multiplicative improvements arrive at a rate proportional to a power of current capability. Within this model, randomness leaves the det...
Xiao-Yu Li, Zheng Gao, Chi-Wun Yang et al.· 0 citations
Hyper-connections widen the residual stream of a Transformer to $n$ parallel streams. Their manifold-constrained version (mHC) mixes the streams at each layer with a doubly stochastic matrix, which it computes by Sinkhorn normalization of exponentiated logits. We give a geometric theory of this design on the Birkhoff p...
Large language model (LLM) inference is increasingly moving toward lower precision to realize the throughput of hardware accelerators, but aggressive post-training quantization (PTQ) can degrade model quality. We present QATFactory, an open-source framework for deployment-aligned quantization-aware distillation (QAD) a...
Wei-Li Xu, Ji-Sen Li, Yu-Qing Jian et al.· 1 citation
A spherical-cap construction proves the latter claim without assuming sparsity merely on the sampling support without assuming sparsity merely on the sampling support, and obtains agnostic minimax excess-risk bounds of order up to logarithms.
Xiao-Yu Li, Zhizhou Sha, Jiao-Jiao Jiang et al.· 0 citations
Reinforcement learning with verifiable rewards (RLVR) is rapidly advancing the reasoning capabilities of language models, yet the optimization layer that converts reward feedback into weight-space updates remains poorly understood. Building on our prior analysis (Zhu et al., 2025), we study this missing layer through t...
