Skip to content

When Updating Stops Being Learning: Rethinking LLM Self-Evolution via learnable information gain

Sep 2026 · arXiv.org · Vol abs/2609.36535 · 0 citations · 59 references
Computer Science

TL;DR

A holistic framework based on learnable information gain, which measures how much novel, parameterizable information a round provides relative to the previous round, and proposes ATRI (Adaptive Training Regulation via Information-gain), which reweights samples within a round and halts training across rounds when information gain remains low.

Abstract

Self-evolution lets large language models (LLMs) improve iteratively using their own generated data, but often suffers from self-evolution degeneration: performance improves, plateaus, then declines. Existing methods address this issue at the component level, targeting either the Questioner or the Solver, and overlook that self-evolution is a tightly coupled system. We propose a holistic framework based on learnable information gain, which measures how much novel, parameterizable information a round provides relative to the previous round. Theoretically, this gain equals the Kullback-Leibler divergence between the two rounds'data distributions plus their entropy change. Practically, it is estimated by fitting a small language model to the previous round and scoring new data via negative log-likelihood. Based on this diagnostic, we propose ATRI (Adaptive Training Regulation via Information-gain), which reweights samples within a round and halts training across rounds when information gain remains low. Experiments on popular datasets demonstrate the superiority of our proposal.

View source

Similar papers

UnlearningSoup: Is Repeated Tuning Necessary for Large Language Model Unlearning?

Whether unlearning runs exhibit exploitable structure in weight space is investigated, and it is observed that models from different runs still lie in a shared evaluation-performance basin, suggesting that stronger models may be recovered through an unlearning-tailored soup strategy, reducing the need for repeated tuni...

Pu-Ning Yang, Qi-Zhou Wang, Jun-Chi Yu et al. · 0 citations

Continual Learning via Self-Probe Gradients

The method, CPLUS, uses self-probe and past-sample gradients to scale down parameter updates that conflict with prior behavior, and becomes more effective as models grow: within the Qwen3 model family, it recovers an increasing share of the forgetting caused by standard fine-tuning.

Dongkyu Cho, R. Chunara, Sung-Min Cha · 1 citation
#artificial intelligence Preprint Sep 2026

COEVO: Co-Evolving Context and Parameters for Recursive Self-Improvement

COEVO is introduced, a framework that updates model parameters from on-policy experience while adapting contextual guidance according to the state of the evolving policy, and suggests that external context should be viewed not merely as a fixed interface to a large language model, but as an adaptive component of recurs...

Si-Wei Chen, Xin-Ping Bao, Xin-Yu Cai et al. · 1 citation

Time-Incremental Continued Pretraining of LLMs: Knowledge Updates Without Catastrophic Forgetting

This work evaluates continued pretraining on FineWeb-Edu dumps drawn strictly from after each model's knowledge cutoff, evaluated across six open-weight models spanning three families and four parameter scales, painting a more optimistic picture of time-incremental CPT than the prior continual learning literature sugge...

Firat Öncel, S. Ali, M. Ravanelli et al. · 0 citations
Book Open access Aug 2026

ANCHOR: Taming Entropy Dynamics for Stable and Efficient Reasoning of Large Language Models

It is shown that policy entropy bounds both the policy gradient and probability update norms; consequently, entropy collapse effectively stops reward signal backpropagation, preventing further policy learning regardless of data quality.

Cong Qin, Jiaye Lin, Xiaoliang Fu et al. · 0 citations

Related blog posts

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.