(c) 2026 Pranay Mahendrakar. Licensed under CC BY 4.0. Exploration bonuses have returned. Between 2025 and 2026 at least six frameworks added an intrinsic novelty or uncertainty term to reinforcement learning with verifiable rewards for language models, each naming a classical antecedent - prediction error, pseudo-counts, epistemic uncertainty - and each reporting gains. The classical literature those antecedents come from also contains a controlled reassessment. In work published at ICLR 2020, a study held the learning algorithm fixed, tuned every bonus, and compared against plain undirected exploration across the full Atari suite; it reported that bonuses beat the simple scheme on one celebrated game, showed no visible difference from it on the rest of the designated hard-exploration set, and never beat it on games where exploration is not the bottleneck. This paper states what that reassessment establishes and, at comparable length, what it does not; extracts the six design conditions that made it informative; and audits the language-model revival against them. Sixteen papers in the revival were checked mechanically for a citation to it, and none contains one. Four of the six conditions are met by at least one paper in the revival and a fifth in part. The condition the reassessment was built to test - an evaluation arm where exploration is not the bottleneck - is met by none of them in the form it requires, although the pattern that condition exists to detect is already visible in one revival paper's own published table. This paper reports no experiments. It names two failed direct imports that the revival itself reports and does not read as evidence about transfer, states where the analogy breaks on the substrate rather than on the evidence, and specifies the comparison a 2026 survey independently asks for without knowing it has been run once already. The literature search, drafting and citation verification for this paper were carried out with AI assistance under the author's direction. Every citation was machine-verified against the arXiv record before inclusion, and every quantitative claim was read back against the cited source's own abstract or, where a claim is drawn from a paper's body or a published table, against the located passage. The absence claims in Section 7 were produced by fetching each named paper's full text and searching it mechanically; the procedure is stated in Section 2 so that it can be repeated. The author is responsible for the final text and for all claims made in it.
Pranay M. Mahendrakar· Zenodo (CERN European Organi...· 0 citations
(c) 2026 Pranay Mahendrakar. Licensed under CC BY 4.0. Exploration bonuses have returned. Between 2025 and 2026 at least six frameworks added an intrinsic novelty or uncertainty term to reinforcement learning with verifiable rewards for language models, each naming a classical antecedent - prediction error, pseudo-counts, epistemic uncertainty - and each reporting gains. The classical literature those antecedents come from also contains a controlled reassessment. In work published at ICLR 2020, a study held the learning algorithm fixed, tuned every bonus, and compared against plain undirected exploration across the full Atari suite; it reported that bonuses beat the simple scheme on one celebrated game, showed no visible difference from it on the rest of the designated hard-exploration set, and never beat it on games where exploration is not the bottleneck. This paper states what that reassessment establishes and, at comparable length, what it does not; extracts the six design conditions that made it informative; and audits the language-model revival against them. Sixteen papers in the revival were checked mechanically for a citation to it, and none contains one. Four of the six conditions are met by at least one paper in the revival and a fifth in part. The condition the reassessment was built to test - an evaluation arm where exploration is not the bottleneck - is met by none of them in the form it requires, although the pattern that condition exists to detect is already visible in one revival paper's own published table. This paper reports no experiments. It names two failed direct imports that the revival itself reports and does not read as evidence about transfer, states where the analogy breaks on the substrate rather than on the evidence, and specifies the comparison a 2026 survey independently asks for without knowing it has been run once already. The literature search, drafting and citation verification for this paper were carried out with AI assistance under the author's direction. Every citation was machine-verified against the arXiv record before inclusion, and every quantitative claim was read back against the cited source's own abstract or, where a claim is drawn from a paper's body or a published table, against the located passage. The absence claims in Section 7 were produced by fetching each named paper's full text and searching it mechanically; the procedure is stated in Section 2 so that it can be repeated. The author is responsible for the final text and for all claims made in it.
Pranay M. Mahendrakar· Zenodo (CERN European Organi...· 0 citations
(c) 2026 Pranay Mahendrakar. Licensed under CC BY 4.0. Memory systems for language-model agents almost all contain a step called consolidation, and almost all of them cite, or gesture at, the complementary learning systems account of hippocampus and neocortex when they name it. In that account consolidation is a specific operation: repeated replay from a fast, sparsely coded store into a slow learner whose shared parameters change, which is what produces generalisation to material never stored and which is also where interference lives. This paper separates what that theory commits its borrower to from what agent memory systems actually do. Four commitments are stated and used as an audit instrument. Against them, deployed agent memory divides into two families and neither instantiates the mechanism, for opposite reasons. The larger, textual family changes no parameters at all: its consolidation is iterated LLM-authored rewriting of an external store, and a 2026 controlled study reports that iterating it drives utility up and then down, in their setting below the no-memory baseline, while no replay result located here reports falling below its own no-replay control. A smaller parametric family, which appeared during 2026 and falsifies the common claim that agent memory never touches weights, does change parameters, but most of it buys stability through per-task adapter isolation or expandable blocks, and isolation withholds the shared representation that the source theory identifies as the common cause of interference and generalisation alike. The paper argues that the field's avoidance of online parametric transfer is well supported by evidence about what such transfer costs, and that what is not supported is retaining the vocabulary while declining the mechanism. It states the five measurements that would decide the question and identifies the single published configuration whose shape matches the theory. The literature search, drafting and citation verification for this paper were carried out with AI assistance under the author's direction. Every citation was machine-verified against the arXiv API and Crossref before inclusion, and every quantitative claim was read back against the cited source's own abstract or, where a claim is drawn from a paper's body, against the located passage. The author is responsible for the final text and for all claims made in it.
Pranay M. Mahendrakar· Zenodo (CERN European Organi...· 0 citations
(c) 2026 Pranay Mahendrakar. Licensed under CC BY 4.0. Memory systems for language-model agents almost all contain a step called consolidation, and almost all of them cite, or gesture at, the complementary learning systems account of hippocampus and neocortex when they name it. In that account consolidation is a specific operation: repeated replay from a fast, sparsely coded store into a slow learner whose shared parameters change, which is what produces generalisation to material never stored and which is also where interference lives. This paper separates what that theory commits its borrower to from what agent memory systems actually do. Four commitments are stated and used as an audit instrument. Against them, deployed agent memory divides into two families and neither instantiates the mechanism, for opposite reasons. The larger, textual family changes no parameters at all: its consolidation is iterated LLM-authored rewriting of an external store, and a 2026 controlled study reports that iterating it drives utility up and then down, in their setting below the no-memory baseline, while no replay result located here reports falling below its own no-replay control. A smaller parametric family, which appeared during 2026 and falsifies the common claim that agent memory never touches weights, does change parameters, but most of it buys stability through per-task adapter isolation or expandable blocks, and isolation withholds the shared representation that the source theory identifies as the common cause of interference and generalisation alike. The paper argues that the field's avoidance of online parametric transfer is well supported by evidence about what such transfer costs, and that what is not supported is retaining the vocabulary while declining the mechanism. It states the five measurements that would decide the question and identifies the single published configuration whose shape matches the theory. The literature search, drafting and citation verification for this paper were carried out with AI assistance under the author's direction. Every citation was machine-verified against the arXiv API and Crossref before inclusion, and every quantitative claim was read back against the cited source's own abstract or, where a claim is drawn from a paper's body, against the located passage. The author is responsible for the final text and for all claims made in it.
Pranay M. Mahendrakar· Zenodo (CERN European Organi...· 0 citations
(c) 2026 Pranay Mahendrakar. Licensed under CC BY 4.0. Reinforcement learning with verifiable rewards is the standard route to reasoning-tuned language models, and the field has split over whether its gains leave the training domain. One body of work reports that most models succeeding at mathematics fail to transfer and that single-domain post-training yields no statistically significant out-of-domain improvement; another reports that post-training on constraint-satisfaction puzzles alone raises hard-mathematics accuracy substantially. This paper argues that the two are not in direct contradiction, because the word transfer carries at least four logically independent quantities: whether training on a domain improves others, whether a domain improves when others are trained, whether a domain is preserved rather than degraded, and whether joint training beats separate training. Papers measuring one are routinely cited as evidence about another. The paper then argues that the reconciliation the field has settled on - that transfer follows structural similarity between source and target - is contradicted by the directional findings of the negative result usually cited for it, which reports unstructured domains transferring to structured ones while failing to transfer to each other. Similarity is symmetric; the reported relation is not, and multi-task learning has treated directed, sign-bearing task-affinity matrices as its normal object of study since Taskonomy. Two further problems are set out: the source-side gain may be substantially elicitation of a pretraining-frequent behaviour rather than acquired skill, which would relocate transfer to the base model, and the reported effect sizes sit near a documented seed-to-seed noise floor. What is not known is stated flatly: no published experiment reports a full directed transfer matrix over a fixed domain set, one protocol and more than one model family. Five measurements that would settle the open part are specified. The literature search, drafting and citation verification for this paper were carried out with AI assistance under the author's direction. Every citation was machine-verified against the arXiv API and Crossref before inclusion, and every quantitative claim was read back against the cited source's own abstract or, where a claim is drawn from a paper's body, against the located passage. The author is responsible for the final text and for all claims made in it.
Pranay M. Mahendrakar· Zenodo (CERN European Organi...· 0 citations
(c) 2026 Pranay Mahendrakar. Licensed under CC BY 4.0. The literature on how language models behave when retrieved context contradicts parametric memory contains a flat contradiction, and neither of its two surveys resolves it. One body of results reports over-reliance on memorised information; another reports that models are highly receptive to conflicting external evidence; a third reports that knowledge updates fail less often than previously published numbers imply. This paper argues that the contradiction is largely not a disagreement about models, because the studies do not share a measurand. A context-memory conflict is not an event that is observed; it is an object the experimenter builds, and seven design choices go into building it: what the conflicting passage is made of, which side is stipulated to be correct and whether the model's belief was elicited or assumed, what quantity is measured, which items are in the sample, what the task demands, what the prompt says, and which model was tested and what was done to it after pretraining. For five of the seven, a single published study varies that choice while holding the others fixed and the reported behaviour moves with it; for the remaining two the evidence is a comparison across studies and is labelled as such. The strongest available adjudication is a 2026 reproducibility study that ran two benchmarks with opposite conclusions under each other's evaluation protocol and attributed the outcome to dataset design, evaluation metric and model size. The reading offered here is narrower than the one the field is converging on. It is not that task demand is the discriminating variable, which one careful study established for one variable while holding others constant, but that the moderators replicate and the point estimates do not: prior confidence, context plausibility, entity frequency and internal inconsistency recur across studies as moderators of context adoption, several of them with a consistent sign, while the adoption rate itself is set by the construction. What follows is that no published number in this literature has been shown to identify a model-level disposition, that a paper reporting only such a number cannot be compared to another, and that the reporting needed to make them comparable is small and is mostly not being done. The literature search, drafting and citation verification for this paper were carried out with AI assistance under the author's direction. Every citation was machine-verified against the arXiv API and Crossref before inclusion, and every quantitative claim was read back against the cited source's own abstract. The author is responsible for the final text and for all claims made in it.
Pranay M. Mahendrakar· Zenodo (CERN European Organi...· 0 citations
(c) 2026 Pranay Mahendrakar. Licensed under CC BY 4.0. The literature on how language models behave when retrieved context contradicts parametric memory contains a flat contradiction, and neither of its two surveys resolves it. One body of results reports over-reliance on memorised information; another reports that models are highly receptive to conflicting external evidence; a third reports that knowledge updates fail less often than previously published numbers imply. This paper argues that the contradiction is largely not a disagreement about models, because the studies do not share a measurand. A context-memory conflict is not an event that is observed; it is an object the experimenter builds, and seven design choices go into building it: what the conflicting passage is made of, which side is stipulated to be correct and whether the model's belief was elicited or assumed, what quantity is measured, which items are in the sample, what the task demands, what the prompt says, and which model was tested and what was done to it after pretraining. For five of the seven, a single published study varies that choice while holding the others fixed and the reported behaviour moves with it; for the remaining two the evidence is a comparison across studies and is labelled as such. The strongest available adjudication is a 2026 reproducibility study that ran two benchmarks with opposite conclusions under each other's evaluation protocol and attributed the outcome to dataset design, evaluation metric and model size. The reading offered here is narrower than the one the field is converging on. It is not that task demand is the discriminating variable, which one careful study established for one variable while holding others constant, but that the moderators replicate and the point estimates do not: prior confidence, context plausibility, entity frequency and internal inconsistency recur across studies as moderators of context adoption, several of them with a consistent sign, while the adoption rate itself is set by the construction. What follows is that no published number in this literature has been shown to identify a model-level disposition, that a paper reporting only such a number cannot be compared to another, and that the reporting needed to make them comparable is small and is mostly not being done. The literature search, drafting and citation verification for this paper were carried out with AI assistance under the author's direction. Every citation was machine-verified against the arXiv API and Crossref before inclusion, and every quantitative claim was read back against the cited source's own abstract. The author is responsible for the final text and for all claims made in it.
Pranay M. Mahendrakar· Zenodo (CERN European Organi...· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.
