Author

Zheng-Zhong Liu

6 papers indexed here

We haven’t gathered this author’s papers yet. Follow them and we’ll fetch their work.

Not the right person? Other researchers publish under this name.

#machine learning Preprint Oct 2026

Towards Looped Models Done Right, Part I: Topology, Input Injection, Recurrent-State Design

Looped language models reuse physical Transformer blocks across logical depth, but established architectures entangle recurrence placement, input injection, and recurrent state organization. We isolate these axes in models trained from scratch at matched parameter scale, logical depth, and token budgets, and then test...

Benhao Huang, Chu-Fan Shi, Jun-Lin Chen et al. · 4 citations

#artificial intelligence Preprint Oct 2026

Taming Softmax at the Limits of Muon: A Unified Approach through Hilbert Geometry

Muon controls weight updates through the RMS movement of a linear layer's output. This implicitly treats the linear output as a proxy for the full module, including the nonlinearity that follows. This proxy can be misleading for softmax, where the RMS shift in activations does not faithfully measure the change in the o...

Aditya Somasundaram, Charles Guille-Escuret, Alexander Moreno et al. · 0 citations

#machine learning Preprint Oct 2026

Towards Looped Models Done Right, Part II: Rethinking at Fixed Points

Every recurrence of a looped language model adds cost in training, decoding, prefill, and reinforcement learning (RL). The closer recurrent states get to fixed points, the less the path to them matters. This enables truncated backpropagation in training; terminal key-value (KV) sharing for decoding with almost no loss...

Benhao Huang, Chu-Fan Shi, Jun-Lin Chen et al. · 1 citation

Preprint Aug 2026

Toward a First-Principles Update Geometry for the Language-Model Head

Muon motivates designing optimizer geometry around the function of each parameter block and uses the spectral norm for hidden linear layers. For the language-model head, the spectral norm is not a faithful measure of functional change. Softmax removes shared logit shifts, whereas the spectral norm can assign arbitraril...

Aditya Somasundaram, Charles Guille-Escuret, Alexander Moreno et al. · 0 citations

#artificial intelligence Preprint Sep 2026

Magenta: Closing the Loop Between Mathematical Reasoning and Lean Verification

This paper introduces Magenta, a training-free agentic pipeline that, given only a natural-language problem, produces an answer, expresses it as a Lean 4 statement, and constructs a machine-checked proof, which achieves 100% accuracy across all evaluated olympiad benchmarks.

Joshua Ong Jun Leang, Hao-Nan Li, Zheng-Yang Zhao et al. · 0 citations

#machine learning Preprint Sep 2026

Unlocking Lossless Speedups in LLMs via Discrete Diffusion

Large Language Models (LLMs) owe much of their success to next-token prediction (NTP), but their autoregressive (AR) structure requires slow, sequential token generation. To overcome this bottleneck, we introduce diffusion-augmented LLMs, a new class of models that defines an AR model distribution while using diffusion...

S. Sahoo, Ling-Jie Chen, Khiem Pham et al. · 1 citation