This Zenodo record is a permanently preserved version of a PREreview. You can view the complete PREreview at https://prereview.org/reviews/22775771. Summary The authors tackle a real obstacle in interpretability: you can't compare SAE features across training checkpoints, because separately trained dictionaries assign feature indices arbitrarily. Their fix is simple and sensible — pool activations from the base model and every RL checkpoint, train one SAE on the lot, freeze it, and re-encode each checkpoint through that fixed dictionary. Applied to GRPO on Qwen2.5-1.5B-Instruct, they find RL changes surprisingly little: drift is small, gradual, concentrated in late layers, and centered on about ten features that fire on formatting scaffolds — step breaks, LaTeX delimiters, answer wrappers — rather than problem content. Steer those directions into the base model and you get most of RL's measured gain back. Introduction Clear. The setup works well: behavioral accounts of RL contradict each other, all of them grade outputs rather than looking inside, SAEs are the obvious tool, and existing SAE methods can't handle checkpoints. The objective follows naturally — what does RL give a model at the representation level, and can it create genuinely novel features? Methods — somewhat appropriate The core idea is sound, and the appendix is unusually thorough: four RL algorithms, five SAE architectures, three model scales, placebo feature sets, random-direction controls, and a two-grader protocol whose decomposition is actually validated in Table 3 rather than assumed. Four things hold it back: The synthetic benchmark is missing. The abstract and introduction both promise a benchmark with features known by construction, to test whether RL instills novel features. It isn't in Section 4 or anywhere in the appendix. This isn't a detail — novel-feature detection is half the paper's stated question and the main reason the pooled design is supposed to beat SAE Track. FVU sits around 0.52. Half the activation variance is unexplained. The authors rightly note it's stable, which rules out dictionary degeneration, but never ask whether reconstruction this weak supports claims about specific feature identity. Amplified features replicate worse than the framing suggests. Cross-algorithm agreement is reported as Spearman 0.42, but top-K Jaccard is 0.28 for amplified features versus 0.61 for suppressed. The amplified set carries the headline finding and the whole steering intervention. Causal claims rest on one 1.5B model. The authors say plainly that ablations are inconclusive at 3B and 7B. Honest, but limiting. Conclusions — somewhat supported The main finding holds up from several directions at once. The formatting signature appears whether features are ranked by endpoint drift or by reward correlation. The elicitation claim rests on three independent results: steering reproduces the effect, all 29 RL-fixed problems are base-solvable within 64 samples, and few-shot prompting substitutes for RL on the base model while adding nothing to the RL model. The authors also qualify carefully. They don't claim RL never improves reasoning — sustained training does help. And their handling of the reasoning set R is the best reasoning in the paper: it's causally necessary but can't be injected as a static vector, and rather than smoothing that over they conclude reasoning depends on dynamically sustained computation. Two problems: "Around 80%" doesn't come from anywhere. At α = 10, steering gives strict 0.192 and tag-rate 0.600 against RL-final's 0.175 and 0.533. Steering exceeds RL on both. No calculation in the paper yields 80%, and the conclusion says "most" instead. Steering rests on n = 120 with overlapping intervals. The α = 10 strict interval is [.131, .271], RL-final's is [.117, .253]. "Matches within confidence intervals" is true but thinner than it sounds, especially since the steering features were selected on the same trajectory they're tested against. Figures — somewhat appropriate and clear Figure 3 is the paper's best: the inverted-U with RL reference lines shows the match and the off-manifold collapse in one glance. Table 7's Wilson intervals are the right call, and Table 3 lets a reader audit the grader decomposition rather than trust it. But several figures are unreadable at page size. Figure 7 is the worst — three panels with legends listing dozens of run names in microscopic type, conveying essentially nothing. Figure 2's 24 stacked labels, Figure 6's overlapping trajectories, and Figure 9's axis labels all have the same problem. Discussion — somewhat clear Strong where it explains mechanism. The timing result (86% of amplification done within 50 steps) turns the elicitation story into something with a shape. Re-characterizing the apparent collapse as reward-hacking brevity — checked against held-out generation length and R's stability — is the right diagnostic instinct. And keeping the superseded steering analysis in A.12, with an explanation of why it misled, is a level of transparency most papers skip. Weak where it should consolidate. The conclusion recaps without revisiting the reproducibility asymmetry, the inconclusive larger-scale results, or the missing benchmark. There's no limitations section, and next steps amount to "future work at larger scales" — not even the benchmark they already promised. Contribution — somewhat likely The method is genuinely reusable and generalizes past RL to any training-trajectory analysis. The findings land in a live dispute, and the convergent evidence for elicitation is more convincing than any single result would be. What limits it: one small model, one reward function that explicitly requires an answer wrapper — so "RL amplifies formatting" may partly be a finding about this reward rather than RL — and the unanswered half of the framing question. Language Clear and precise. Two things to fix: the Acknowledgments section is empty, and A.13 has an unresolved citation ("the unbiased estimator of ?"). Minor slips elsewhere ("RL genuinely incentivize," "limitedly understood") don't impede reading. Recommendation Worth reading if you work on interpretability of training dynamics or on what RLVR actually does. The method is useful and the elicitation evidence is well triangulated. Three things before it's ready: Include the synthetic benchmark or drop the claim. Right now the abstract advertises an experiment the paper doesn't contain. Fix or remove "around 80%." Steering beats RL-final on two of three metrics; whatever 80% refers to needs deriving. Add a limitations section and a conclusion that folds in the reproducibility asymmetry and the scale-limited causal evidence. Larger n for steering, held-out feature selection, and an RL run whose reward doesn't require a format wrapper would all strengthen it further. Competing interests The author declares that they have no competing interests. Use of Artificial Intelligence (AI) The author declares that they used generative AI to come up with new ideas for their review.
Sneh Pankajbhai Vora· Zenodo (CERN European Organi...· 0 citations
This Zenodo record is a permanently preserved version of a PREreview. You can view the complete PREreview at https://prereview.org/reviews/22775771. Summary The authors tackle a real obstacle in interpretability: you can't compare SAE features across training checkpoints, because separately trained dictionaries assign feature indices arbitrarily. Their fix is simple and sensible — pool activations from the base model and every RL checkpoint, train one SAE on the lot, freeze it, and re-encode each checkpoint through that fixed dictionary. Applied to GRPO on Qwen2.5-1.5B-Instruct, they find RL changes surprisingly little: drift is small, gradual, concentrated in late layers, and centered on about ten features that fire on formatting scaffolds — step breaks, LaTeX delimiters, answer wrappers — rather than problem content. Steer those directions into the base model and you get most of RL's measured gain back. Introduction Clear. The setup works well: behavioral accounts of RL contradict each other, all of them grade outputs rather than looking inside, SAEs are the obvious tool, and existing SAE methods can't handle checkpoints. The objective follows naturally — what does RL give a model at the representation level, and can it create genuinely novel features? Methods — somewhat appropriate The core idea is sound, and the appendix is unusually thorough: four RL algorithms, five SAE architectures, three model scales, placebo feature sets, random-direction controls, and a two-grader protocol whose decomposition is actually validated in Table 3 rather than assumed. Four things hold it back: The synthetic benchmark is missing. The abstract and introduction both promise a benchmark with features known by construction, to test whether RL instills novel features. It isn't in Section 4 or anywhere in the appendix. This isn't a detail — novel-feature detection is half the paper's stated question and the main reason the pooled design is supposed to beat SAE Track. FVU sits around 0.52. Half the activation variance is unexplained. The authors rightly note it's stable, which rules out dictionary degeneration, but never ask whether reconstruction this weak supports claims about specific feature identity. Amplified features replicate worse than the framing suggests. Cross-algorithm agreement is reported as Spearman 0.42, but top-K Jaccard is 0.28 for amplified features versus 0.61 for suppressed. The amplified set carries the headline finding and the whole steering intervention. Causal claims rest on one 1.5B model. The authors say plainly that ablations are inconclusive at 3B and 7B. Honest, but limiting. Conclusions — somewhat supported The main finding holds up from several directions at once. The formatting signature appears whether features are ranked by endpoint drift or by reward correlation. The elicitation claim rests on three independent results: steering reproduces the effect, all 29 RL-fixed problems are base-solvable within 64 samples, and few-shot prompting substitutes for RL on the base model while adding nothing to the RL model. The authors also qualify carefully. They don't claim RL never improves reasoning — sustained training does help. And their handling of the reasoning set R is the best reasoning in the paper: it's causally necessary but can't be injected as a static vector, and rather than smoothing that over they conclude reasoning depends on dynamically sustained computation. Two problems: "Around 80%" doesn't come from anywhere. At α = 10, steering gives strict 0.192 and tag-rate 0.600 against RL-final's 0.175 and 0.533. Steering exceeds RL on both. No calculation in the paper yields 80%, and the conclusion says "most" instead. Steering rests on n = 120 with overlapping intervals. The α = 10 strict interval is [.131, .271], RL-final's is [.117, .253]. "Matches within confidence intervals" is true but thinner than it sounds, especially since the steering features were selected on the same trajectory they're tested against. Figures — somewhat appropriate and clear Figure 3 is the paper's best: the inverted-U with RL reference lines shows the match and the off-manifold collapse in one glance. Table 7's Wilson intervals are the right call, and Table 3 lets a reader audit the grader decomposition rather than trust it. But several figures are unreadable at page size. Figure 7 is the worst — three panels with legends listing dozens of run names in microscopic type, conveying essentially nothing. Figure 2's 24 stacked labels, Figure 6's overlapping trajectories, and Figure 9's axis labels all have the same problem. Discussion — somewhat clear Strong where it explains mechanism. The timing result (86% of amplification done within 50 steps) turns the elicitation story into something with a shape. Re-characterizing the apparent collapse as reward-hacking brevity — checked against held-out generation length and R's stability — is the right diagnostic instinct. And keeping the superseded steering analysis in A.12, with an explanation of why it misled, is a level of transparency most papers skip. Weak where it should consolidate. The conclusion recaps without revisiting the reproducibility asymmetry, the inconclusive larger-scale results, or the missing benchmark. There's no limitations section, and next steps amount to "future work at larger scales" — not even the benchmark they already promised. Contribution — somewhat likely The method is genuinely reusable and generalizes past RL to any training-trajectory analysis. The findings land in a live dispute, and the convergent evidence for elicitation is more convincing than any single result would be. What limits it: one small model, one reward function that explicitly requires an answer wrapper — so "RL amplifies formatting" may partly be a finding about this reward rather than RL — and the unanswered half of the framing question. Language Clear and precise. Two things to fix: the Acknowledgments section is empty, and A.13 has an unresolved citation ("the unbiased estimator of ?"). Minor slips elsewhere ("RL genuinely incentivize," "limitedly understood") don't impede reading. Recommendation Worth reading if you work on interpretability of training dynamics or on what RLVR actually does. The method is useful and the elicitation evidence is well triangulated. Three things before it's ready: Include the synthetic benchmark or drop the claim. Right now the abstract advertises an experiment the paper doesn't contain. Fix or remove "around 80%." Steering beats RL-final on two of three metrics; whatever 80% refers to needs deriving. Add a limitations section and a conclusion that folds in the reproducibility asymmetry and the scale-limited causal evidence. Larger n for steering, held-out feature selection, and an RL run whose reward doesn't require a format wrapper would all strengthen it further. Competing interests The author declares that they have no competing interests. Use of Artificial Intelligence (AI) The author declares that they used generative AI to come up with new ideas for their review.
Sneh Pankajbhai Vora· Zenodo (CERN European Organi...· 0 citations
This Zenodo record is a permanently preserved version of a Structured PREreview. You can view the complete PREreview at https://prereview.org/reviews/22662419. Does the introduction explain the objective of the research presented in the preprint? Yes The introduction explains the objective by setting up a gap and then posing a question. It moves in 4 steps: 1. Drug discovery means searching a huge chemical space for molecules meeting competing objectives. Scoring oracles range from cheap and low-fidelity to expensive and high-fidelity, and that cost limits how much space gets searched. 2. LLMs are a good fit, but existing RLVR work trains only on cheap oracles — and training directly on expensive ones is impractical because online RL becomes too slow or produces stale rollouts. 3. So the question: can LLMs be post-trained for expensive, high-fidelity objectives by scaling training on cheaper synthetic tasks instead? 4. The objective is phrased as a question ("we investigate whether") rather than an aim. The key phrase is "too expensive to train on" — it names the constraint that makes the question worth asking and points straight at the synthetic-task solution. Are the methods well-suited for this research? Somewhat appropriate 1. The frontier comparison isn't fair. Trained models had 500 steps of practice with the exact prompt format. Frontier models got none. Table A3 shows Claude Opus 4.8 failed formatting in 39 of 200 rollouts, GLM-5.2 in 71 — and every failure counts as zero improvement. Part of the reported gap is formatting, not chemistry. No few-shot baseline separates the two. 2. The error bars are too wide for the rankings. The top result is +3.88 ± 1.98 kcal/mol. Twelve recipes are compared, each with one training seed, 40 systems, five rollouts. Change the reward weights and the ordering reshuffles (Figures 3, A3, A4). The broad claim holds; specific claims like "T2 beats Mix at 300 steps" don't. 3. One base model, one metric. Everything starts from Qwen3.6-35B-A3B, so the curriculum effect may not generalize to other models. And SQM2.20 is the only measure of binding — no FEP or experimental check. Trained models strip anionic groups for a mean 26.1 kcal/mol gain, which could be real chemistry or an artifact of the scoring function. Nothing distinguishes them. 4. Small asymmetry. The scaffold-checking tool is given at evaluation but withheld during training. Are the conclusions supported by the data? Somewhat supported The main claim is solid: cheap synthetic tasks transfer to an expensive held-out objective. Every curriculum model beats T1(500), and the T2(500) collapse shows the curriculum is doing real work. The efficiency gains (7.7 turns vs 13–18) are large enough that noise doesn't threaten them. The authors are also honest where it costs them — they state plainly that no scaling law appears, and they flag T1(250)–Mix(250) as an exception to their own mixing conclusion. Where they reach past the data: 1. "Surpasses frontier models." Their own Table A3 shows Opus 4.8 failing formatting in 39/200 rollouts, GLM-5.2 in 71, each scored as zero. The data is reported but never qualifies the claim. 2. Recipe rankings stated too firmly. "No clear benefit to mixing" rests on ±1.98 kcal/mol error bars, one training seed each, and an ordering that reshuffles under different reward weights. 3. Chemistry read as strategy. Exploring worse intermediates is called "encouraging" exploration; it's equally consistent with noisy search. Anionic-group removal worth 26.1 kcal/mol is treated as a learned principle, but could be an SQM scoring artifact. Nothing tests which. 4. Wet-lab extension. The conclusion suggests the approach may transfer to experimental tasks. Hedged, but this study is entirely in silico. Net: reasonable interpretations throughout, with a few claims carrying more confidence than single seeds, wide error bars, and one scoring function can support. Are the data presentations, including visualizations, well-suited to represent the data? Somewhat appropriate and clear What works. Figure 1 carries the paper. The three-tier schematic with real prompt text makes the task hierarchy immediately legible, which matters most since task design is the contribution. Table 2 pairs the headline metric with turns, SQM calls, and constraint satisfaction, so efficiency and affinity can be read together. Table A3's failure breakdown is transparent about where every rollout went. Figure A6 uses a log scale for a wide range and marks unavailable closed-source reasoning as "∼" rather than zero. Error bars appear with their definition stated. What holds it back. 1. Sign convention. Lower SQM means tighter binding, but "relative BA improvement" is built so higher is better. Figure 5 then labels an improvement "ΔSQM = +5.5 kcal/mol." Every caption has to restate "higher is better" — a fix, but also a sign the convention isn't self-evident. 2. Figure 3, right panel. Twelve points, error bars omitted "for clarity." Since the y-differences are about the size of the SEs shown in the left panel, this makes separation look cleaner than it is. 3. Figure 2, right panel. Five overlapping curve pairs in similar greens and oranges, distinguished by dashed vs solid. Telling T1(350)–T2(150) from T1(350)–Mix(150) is genuinely difficult. 4. Color accessibility. The green/orange trained-vs-zero-shot split (Figures 3, A3, A4, A8) is the standard deuteranopia problem. Figure 7 uses a red-green diverging heatmap — the least accessible diverging choice — and suppresses values below 10%, so most cells are readable only by color. 5. Crowding. Figure 6's leader lines converge on small labels near the origin. Figure A10's rotated labels are hard to read and ordered by value rather than recipe family. Net: the tables are strong and the schematic is excellent, but the color palettes, the sign convention, and Figure 3's missing error bars add friction and slightly oversell the pattern. How clearly do the authors discuss, explain, and interpret their findings and potential next steps for the research? Somewhat clearly Clear where they explain mechanism: The T2(500) collapse gets its own appendix with reward-decomposition curves showing the policy saturating constraints while making zero binding progress — that explains why the curriculum is needed, not just that it works. Design choices come with reasons: the tool-penalty warmup is justified by what failed without it, and withholding the scaffold tool during training by the mode collapse it caused. They also state plainly that no scaling law appears, and flag T1(250)–Mix(250) as an exception to their own conclusion. Next steps are concrete: turn-level rewards, supervised finetuning, on-policy distillation. Less clear where they should consolidate: The conclusion is one substantive paragraph and mostly recaps. It never revisits the paper's own complications — the formatting failures in Table A3, the reward-weighting sensitivity in C.2, the missing scaling law. Those sit in appendices and stay there. There is no limitations section. Single base model, single scoring function, one seed per recipe — none acknowledged as bounding the claims. A few readings are asserted rather than argued. Worse intermediate designs are called "encouraging" exploration, with no alternative considered. Anionic removal is treated as a learned principle without asking whether it exploits an SQM bias, and no next step proposes checking. Net: strong mechanistic explanation, honest about the negative result, but the discussion doesn't fold its own caveats back into what it concludes. Is the preprint likely to advance academic knowledge? Somewhat likely The advancements are real: It addresses a genuine bottleneck. Chemically meaningful oracles are too slow for online RL — SQM takes tens of minutes per evaluation, FEP takes GPU-hours — which has confined RLVR molecular design to cheap, low-fidelity rewards. Showing that a curriculum over cheap synthetic tasks transfers to an expensive held-out objective offers a concrete way around that, and the pattern should apply wherever the reward you care about is too costly to train on directly. The T2(500) collapse is the most valuable finding. Training directly on the harder task produces a degenerate policy that saturates constraints and never improves binding — with reward decomposition showing exactly how. That converts "curriculum helps" into "curriculum is necessary here," which is mechanism, not just a benchmark number. The efficiency result deserves more attention than it gets: a 35B model competitive at 7.7 turns versus 13–18 for frontier models, using fewer tokens. Cost per design matters in practice. The task construction is reusable — 200,000 verifiable Tier-1 tasks from 116 descriptors, plus the structural textification approach. Why not "highly likely." Curriculum learning and synthetic data scaling are established ideas. The novelty is the application and the specific low-to-high fidelity transfer in chemistry — a meaningful instance of a known principle, not a new principle. The frontier comparison carries much of the paper's rhetorical weight but is confounded by formatting failures the authors report and never reconcile. One base model, one scoring function, one seed per recipe, wide error bars: the finer rankings won't replicate unchanged. And SQM
Sneh Pankajbhai Vora· Zenodo (CERN European Organi...· 0 citations
This Zenodo record is a permanently preserved version of a PREreview. You can view the complete PREreview at https://prereview.org/reviews/22662611. Review of "Training Large Language Models for Small-Molecule Design with Synthetic Task Scaling" Summary This preprint asks whether LLMs can be post-trained for expensive, high-fidelity molecular design objectives by scaling training on cheaper synthetic tasks. The authors construct a two-tier hierarchy of synthetic lead-optimization tasks — cheap RDKit/OpenEye property optimization (Tier-1) and medium-cost Chemgauss4 docking (Tier-2) — and evaluate curriculum recipes on a held-out objective scored with SQM2.20, which is too expensive to train on directly. They report that curriculum-trained 35B models outperform a Tier-1-only baseline and several frontier models. Objective The introduction clearly explains the objective. It builds a gap-then-question structure: drug discovery requires searching a vast chemical space; scoring oracles range from cheap and low-fidelity to expensive and high-fidelity, and that cost limits search; LLMs are a good fit, but existing RLVR work trains only on cheap oracles, and training directly on expensive ones is impractical because online RL becomes too slow or produces stale rollouts. The objective follows as a research question — whether cheap synthetic tasks can substitute — phrased as "we investigate whether," which keeps it falsifiable. The phrase "too expensive to train on" names the constraint that makes the question non-trivial. Methods — somewhat appropriate The design is well matched to the question. Fixing all runs at 500 steps on identical hardware isolates the curriculum variable; T1(500) and T2(500) are the right anchors; PLINDER train/test splits prevent leakage; PoseBusters filtering, baseline normalization, zero-scoring of invalid rollouts, and the reward-weighting sweep in Appendix C.2 are all careful choices. Four issues limit it: The frontier comparison isn't like-for-like: Trained models had 500 steps of practice with the exact prompt format; frontier models had none. Table A3 shows Claude Opus 4.8 failing formatting in 39 of 200 rollouts and GLM-5.2 in 71, each scored as zero improvement. Part of the reported gap is formatting compliance rather than design skill, and no few-shot or format-adapted baseline separates the two. Statistical resolution is thin for the finer claims: The top result is +3.88 ± 1.98 kcal/mol. Twelve recipes are compared, each with a single training seed, 40 systems, five rollouts, and no multiplicity correction. The ordering reshuffles substantially across the three reward-weighting settings (Figures 3, A3, A4), suggesting run-to-run noise is comparable to the differences being interpreted. One base model, one metric: Everything starts from Qwen3.6-35B-A3B, so it is unclear whether the curriculum benefit is a property of the approach or of this checkpoint — particularly given the authors' own observation that the base model's low-halogenation tendency is amplified by training. SQM2.20 is the sole measure of binding, with no orthogonal check. Minor asymmetry: The contains_scaffold tool is available at evaluation but withheld during training. This is explained, but it complicates the comparison. Conclusions — somewhat supported The central claim holds. Every curriculum model beats T1(500), and the T2(500) collapse into a constraint-only policy directly supports the argument that the curriculum is necessary. The efficiency findings (7.7 turns versus 13–18 for frontier models, with lower token counts) are large enough that noise doesn't threaten them. The authors are commendably honest where it costs them: they state plainly that no clear scaling law emerges, and they flag T1(250)–Mix(250) as an exception to their own conclusion about mixing. Four claims reach past the data: "Surpasses frontier models" is never reconciled with the formatting failures the authors themselves document in Table A3. Recipe rankings such as "no clear benefit to mixing" are stated more firmly than single seeds and wide error bars allow. Chemical behavior is read as strategy: Exploration of worse intermediate designs is called "encouraging" evidence of non-greedy search, with no alternative considered. Anionic-group removal, worth a mean 26.1 kcal/mol, is treated as a learned design principle without testing whether it exploits an SQM scoring bias. The wet-lab extension in the conclusion is hedged with "potentially," but nothing in an all-in silico study speaks to experimental transfer. Data presentation — somewhat appropriate and clear Figure 1 carries the paper: the three-tier schematic with real prompt text makes the task hierarchy immediately legible, which matters most since task design is the contribution. Table 2 pairs the headline metric with turns, SQM calls, and constraint satisfaction. Table A3's failure breakdown is transparent. Figure A6 handles a wide dynamic range with a log scale and marks unavailable closed-source reasoning as "∼" rather than zero. Limitations: Sign convention: Lower SQM means tighter binding, but "relative BA improvement" is constructed so higher is better, and Figure 5 labels improvements as "ΔSQM = +5.5 kcal/mol." Nearly every caption must restate "higher is better." Figure 3, right panel omits error bars "for clarity" while plotting differences comparable in size to the SEs shown in the left panel, making separation look cleaner than it is. Figure 2, right panel overlays five curve pairs in similar greens and oranges; distinguishing T1(350)–T2(150) from T1(350)–Mix(150) is difficult. Color accessibility: The green/orange trained-versus-zero-shot encoding (Figures 3, A3, A4, A8) presents the standard deuteranopia problem. Figure 7 uses a red-green diverging heatmap and suppresses values below 10%, leaving most cells readable only by color. Crowding: Figure 6's leader lines converge on small labels near the origin; Figure A10's rotated labels are ordered by value rather than recipe family. Discussion — somewhat clear The mechanistic explanations are strong. Appendix C.1 gives the T2(500) collapse a reward decomposition showing the policy saturating constraints while making no binding progress, which explains why the curriculum is needed rather than merely asserting it. Design choices come with reasoning: the tool-penalty warmup is justified by what failed without it, and withholding the scaffold tool during training by the mode collapse it caused. Next steps are concrete — turn-level rewards, supervised finetuning, on-policy distillation. The conclusion, however, is a single substantive paragraph that mostly recapitulates. It never revisits the paper's own complications — the formatting failures, the reward-weighting sensitivity, the absent scaling law — which remain in appendices. There is no limitations section: single base model, single scoring function, one seed per recipe are nowhere acknowledged as bounding the claims. Contribution — somewhat likely to advance knowledge The work addresses a real bottleneck. SQM takes tens of minutes per evaluation and FEP takes GPU-hours, which has confined RLVR molecular design to cheap rewards. Demonstrating transfer from cheap synthetic tasks to an expensive held-out objective offers a concrete route around that constraint, and the pattern should generalize beyond chemistry. The T2(500) collapse is the most valuable single finding — a mechanistic negative result that converts "curriculum helps" into "curriculum is necessary here." The efficiency result deserves more attention than it receives. The task-construction recipe (200,000 verifiable tasks from 116 descriptors, plus structural textification) is reusable. Against this: curriculum learning and synthetic data scaling are established, so the novelty is the application rather than the principle. The frontier comparison carries much of the rhetorical weight but is confounded. And whether the models learned chemistry or learned SQM's biases remains unresolved — the question that determines whether any of this transfers to real campaigns. Language No editing needed. The writing is clean, terminology is defined at first use, and notation is consistent. The mixed polarity of the SQM sign convention is a presentation choice rather than a language problem, though it does force repeated clarification. Competing interests The author declares that they have no competing interests. Use of Artificial Intelligence (AI) The author declares that they used generative AI to come up with new ideas for their review.
Sneh Pankajbhai Vora· Zenodo (CERN European Organi...· 0 citations
This Zenodo record is a permanently preserved version of a Structured PREreview. You can view the complete PREreview at https://prereview.org/reviews/22662419. Does the introduction explain the objective of the research presented in the preprint? Yes The introduction explains the objective by setting up a gap and then posing a question. It moves in 4 steps: 1. Drug discovery means searching a huge chemical space for molecules meeting competing objectives. Scoring oracles range from cheap and low-fidelity to expensive and high-fidelity, and that cost limits how much space gets searched. 2. LLMs are a good fit, but existing RLVR work trains only on cheap oracles — and training directly on expensive ones is impractical because online RL becomes too slow or produces stale rollouts. 3. So the question: can LLMs be post-trained for expensive, high-fidelity objectives by scaling training on cheaper synthetic tasks instead? 4. The objective is phrased as a question ("we investigate whether") rather than an aim. The key phrase is "too expensive to train on" — it names the constraint that makes the question worth asking and points straight at the synthetic-task solution. Are the methods well-suited for this research? Somewhat appropriate 1. The frontier comparison isn't fair. Trained models had 500 steps of practice with the exact prompt format. Frontier models got none. Table A3 shows Claude Opus 4.8 failed formatting in 39 of 200 rollouts, GLM-5.2 in 71 — and every failure counts as zero improvement. Part of the reported gap is formatting, not chemistry. No few-shot baseline separates the two. 2. The error bars are too wide for the rankings. The top result is +3.88 ± 1.98 kcal/mol. Twelve recipes are compared, each with one training seed, 40 systems, five rollouts. Change the reward weights and the ordering reshuffles (Figures 3, A3, A4). The broad claim holds; specific claims like "T2 beats Mix at 300 steps" don't. 3. One base model, one metric. Everything starts from Qwen3.6-35B-A3B, so the curriculum effect may not generalize to other models. And SQM2.20 is the only measure of binding — no FEP or experimental check. Trained models strip anionic groups for a mean 26.1 kcal/mol gain, which could be real chemistry or an artifact of the scoring function. Nothing distinguishes them. 4. Small asymmetry. The scaffold-checking tool is given at evaluation but withheld during training. Are the conclusions supported by the data? Somewhat supported The main claim is solid: cheap synthetic tasks transfer to an expensive held-out objective. Every curriculum model beats T1(500), and the T2(500) collapse shows the curriculum is doing real work. The efficiency gains (7.7 turns vs 13–18) are large enough that noise doesn't threaten them. The authors are also honest where it costs them — they state plainly that no scaling law appears, and they flag T1(250)–Mix(250) as an exception to their own mixing conclusion. Where they reach past the data: 1. "Surpasses frontier models." Their own Table A3 shows Opus 4.8 failing formatting in 39/200 rollouts, GLM-5.2 in 71, each scored as zero. The data is reported but never qualifies the claim. 2. Recipe rankings stated too firmly. "No clear benefit to mixing" rests on ±1.98 kcal/mol error bars, one training seed each, and an ordering that reshuffles under different reward weights. 3. Chemistry read as strategy. Exploring worse intermediates is called "encouraging" exploration; it's equally consistent with noisy search. Anionic-group removal worth 26.1 kcal/mol is treated as a learned principle, but could be an SQM scoring artifact. Nothing tests which. 4. Wet-lab extension. The conclusion suggests the approach may transfer to experimental tasks. Hedged, but this study is entirely in silico. Net: reasonable interpretations throughout, with a few claims carrying more confidence than single seeds, wide error bars, and one scoring function can support. Are the data presentations, including visualizations, well-suited to represent the data? Somewhat appropriate and clear What works. Figure 1 carries the paper. The three-tier schematic with real prompt text makes the task hierarchy immediately legible, which matters most since task design is the contribution. Table 2 pairs the headline metric with turns, SQM calls, and constraint satisfaction, so efficiency and affinity can be read together. Table A3's failure breakdown is transparent about where every rollout went. Figure A6 uses a log scale for a wide range and marks unavailable closed-source reasoning as "∼" rather than zero. Error bars appear with their definition stated. What holds it back. 1. Sign convention. Lower SQM means tighter binding, but "relative BA improvement" is built so higher is better. Figure 5 then labels an improvement "ΔSQM = +5.5 kcal/mol." Every caption has to restate "higher is better" — a fix, but also a sign the convention isn't self-evident. 2. Figure 3, right panel. Twelve points, error bars omitted "for clarity." Since the y-differences are about the size of the SEs shown in the left panel, this makes separation look cleaner than it is. 3. Figure 2, right panel. Five overlapping curve pairs in similar greens and oranges, distinguished by dashed vs solid. Telling T1(350)–T2(150) from T1(350)–Mix(150) is genuinely difficult. 4. Color accessibility. The green/orange trained-vs-zero-shot split (Figures 3, A3, A4, A8) is the standard deuteranopia problem. Figure 7 uses a red-green diverging heatmap — the least accessible diverging choice — and suppresses values below 10%, so most cells are readable only by color. 5. Crowding. Figure 6's leader lines converge on small labels near the origin. Figure A10's rotated labels are hard to read and ordered by value rather than recipe family. Net: the tables are strong and the schematic is excellent, but the color palettes, the sign convention, and Figure 3's missing error bars add friction and slightly oversell the pattern. How clearly do the authors discuss, explain, and interpret their findings and potential next steps for the research? Somewhat clearly Clear where they explain mechanism: The T2(500) collapse gets its own appendix with reward-decomposition curves showing the policy saturating constraints while making zero binding progress — that explains why the curriculum is needed, not just that it works. Design choices come with reasons: the tool-penalty warmup is justified by what failed without it, and withholding the scaffold tool during training by the mode collapse it caused. They also state plainly that no scaling law appears, and flag T1(250)–Mix(250) as an exception to their own conclusion. Next steps are concrete: turn-level rewards, supervised finetuning, on-policy distillation. Less clear where they should consolidate: The conclusion is one substantive paragraph and mostly recaps. It never revisits the paper's own complications — the formatting failures in Table A3, the reward-weighting sensitivity in C.2, the missing scaling law. Those sit in appendices and stay there. There is no limitations section. Single base model, single scoring function, one seed per recipe — none acknowledged as bounding the claims. A few readings are asserted rather than argued. Worse intermediate designs are called "encouraging" exploration, with no alternative considered. Anionic removal is treated as a learned principle without asking whether it exploits an SQM bias, and no next step proposes checking. Net: strong mechanistic explanation, honest about the negative result, but the discussion doesn't fold its own caveats back into what it concludes. Is the preprint likely to advance academic knowledge? Somewhat likely The advancements are real: It addresses a genuine bottleneck. Chemically meaningful oracles are too slow for online RL — SQM takes tens of minutes per evaluation, FEP takes GPU-hours — which has confined RLVR molecular design to cheap, low-fidelity rewards. Showing that a curriculum over cheap synthetic tasks transfers to an expensive held-out objective offers a concrete way around that, and the pattern should apply wherever the reward you care about is too costly to train on directly. The T2(500) collapse is the most valuable finding. Training directly on the harder task produces a degenerate policy that saturates constraints and never improves binding — with reward decomposition showing exactly how. That converts "curriculum helps" into "curriculum is necessary here," which is mechanism, not just a benchmark number. The efficiency result deserves more attention than it gets: a 35B model competitive at 7.7 turns versus 13–18 for frontier models, using fewer tokens. Cost per design matters in practice. The task construction is reusable — 200,000 verifiable Tier-1 tasks from 116 descriptors, plus the structural textification approach. Why not "highly likely." Curriculum learning and synthetic data scaling are established ideas. The novelty is the application and the specific low-to-high fidelity transfer in chemistry — a meaningful instance of a known principle, not a new principle. The frontier comparison carries much of the paper's rhetorical weight but is confounded by formatting failures the authors report and never reconcile. One base model, one scoring function, one seed per recipe, wide error bars: the finer rankings won't replicate unchanged. And SQM
Sneh Pankajbhai Vora· Zenodo (CERN European Organi...· 0 citations
This Zenodo record is a permanently preserved version of a PREreview. You can view the complete PREreview at https://prereview.org/reviews/22662611. Review of "Training Large Language Models for Small-Molecule Design with Synthetic Task Scaling" Summary This preprint asks whether LLMs can be post-trained for expensive, high-fidelity molecular design objectives by scaling training on cheaper synthetic tasks. The authors construct a two-tier hierarchy of synthetic lead-optimization tasks — cheap RDKit/OpenEye property optimization (Tier-1) and medium-cost Chemgauss4 docking (Tier-2) — and evaluate curriculum recipes on a held-out objective scored with SQM2.20, which is too expensive to train on directly. They report that curriculum-trained 35B models outperform a Tier-1-only baseline and several frontier models. Objective The introduction clearly explains the objective. It builds a gap-then-question structure: drug discovery requires searching a vast chemical space; scoring oracles range from cheap and low-fidelity to expensive and high-fidelity, and that cost limits search; LLMs are a good fit, but existing RLVR work trains only on cheap oracles, and training directly on expensive ones is impractical because online RL becomes too slow or produces stale rollouts. The objective follows as a research question — whether cheap synthetic tasks can substitute — phrased as "we investigate whether," which keeps it falsifiable. The phrase "too expensive to train on" names the constraint that makes the question non-trivial. Methods — somewhat appropriate The design is well matched to the question. Fixing all runs at 500 steps on identical hardware isolates the curriculum variable; T1(500) and T2(500) are the right anchors; PLINDER train/test splits prevent leakage; PoseBusters filtering, baseline normalization, zero-scoring of invalid rollouts, and the reward-weighting sweep in Appendix C.2 are all careful choices. Four issues limit it: The frontier comparison isn't like-for-like: Trained models had 500 steps of practice with the exact prompt format; frontier models had none. Table A3 shows Claude Opus 4.8 failing formatting in 39 of 200 rollouts and GLM-5.2 in 71, each scored as zero improvement. Part of the reported gap is formatting compliance rather than design skill, and no few-shot or format-adapted baseline separates the two. Statistical resolution is thin for the finer claims: The top result is +3.88 ± 1.98 kcal/mol. Twelve recipes are compared, each with a single training seed, 40 systems, five rollouts, and no multiplicity correction. The ordering reshuffles substantially across the three reward-weighting settings (Figures 3, A3, A4), suggesting run-to-run noise is comparable to the differences being interpreted. One base model, one metric: Everything starts from Qwen3.6-35B-A3B, so it is unclear whether the curriculum benefit is a property of the approach or of this checkpoint — particularly given the authors' own observation that the base model's low-halogenation tendency is amplified by training. SQM2.20 is the sole measure of binding, with no orthogonal check. Minor asymmetry: The contains_scaffold tool is available at evaluation but withheld during training. This is explained, but it complicates the comparison. Conclusions — somewhat supported The central claim holds. Every curriculum model beats T1(500), and the T2(500) collapse into a constraint-only policy directly supports the argument that the curriculum is necessary. The efficiency findings (7.7 turns versus 13–18 for frontier models, with lower token counts) are large enough that noise doesn't threaten them. The authors are commendably honest where it costs them: they state plainly that no clear scaling law emerges, and they flag T1(250)–Mix(250) as an exception to their own conclusion about mixing. Four claims reach past the data: "Surpasses frontier models" is never reconciled with the formatting failures the authors themselves document in Table A3. Recipe rankings such as "no clear benefit to mixing" are stated more firmly than single seeds and wide error bars allow. Chemical behavior is read as strategy: Exploration of worse intermediate designs is called "encouraging" evidence of non-greedy search, with no alternative considered. Anionic-group removal, worth a mean 26.1 kcal/mol, is treated as a learned design principle without testing whether it exploits an SQM scoring bias. The wet-lab extension in the conclusion is hedged with "potentially," but nothing in an all-in silico study speaks to experimental transfer. Data presentation — somewhat appropriate and clear Figure 1 carries the paper: the three-tier schematic with real prompt text makes the task hierarchy immediately legible, which matters most since task design is the contribution. Table 2 pairs the headline metric with turns, SQM calls, and constraint satisfaction. Table A3's failure breakdown is transparent. Figure A6 handles a wide dynamic range with a log scale and marks unavailable closed-source reasoning as "∼" rather than zero. Limitations: Sign convention: Lower SQM means tighter binding, but "relative BA improvement" is constructed so higher is better, and Figure 5 labels improvements as "ΔSQM = +5.5 kcal/mol." Nearly every caption must restate "higher is better." Figure 3, right panel omits error bars "for clarity" while plotting differences comparable in size to the SEs shown in the left panel, making separation look cleaner than it is. Figure 2, right panel overlays five curve pairs in similar greens and oranges; distinguishing T1(350)–T2(150) from T1(350)–Mix(150) is difficult. Color accessibility: The green/orange trained-versus-zero-shot encoding (Figures 3, A3, A4, A8) presents the standard deuteranopia problem. Figure 7 uses a red-green diverging heatmap and suppresses values below 10%, leaving most cells readable only by color. Crowding: Figure 6's leader lines converge on small labels near the origin; Figure A10's rotated labels are ordered by value rather than recipe family. Discussion — somewhat clear The mechanistic explanations are strong. Appendix C.1 gives the T2(500) collapse a reward decomposition showing the policy saturating constraints while making no binding progress, which explains why the curriculum is needed rather than merely asserting it. Design choices come with reasoning: the tool-penalty warmup is justified by what failed without it, and withholding the scaffold tool during training by the mode collapse it caused. Next steps are concrete — turn-level rewards, supervised finetuning, on-policy distillation. The conclusion, however, is a single substantive paragraph that mostly recapitulates. It never revisits the paper's own complications — the formatting failures, the reward-weighting sensitivity, the absent scaling law — which remain in appendices. There is no limitations section: single base model, single scoring function, one seed per recipe are nowhere acknowledged as bounding the claims. Contribution — somewhat likely to advance knowledge The work addresses a real bottleneck. SQM takes tens of minutes per evaluation and FEP takes GPU-hours, which has confined RLVR molecular design to cheap rewards. Demonstrating transfer from cheap synthetic tasks to an expensive held-out objective offers a concrete route around that constraint, and the pattern should generalize beyond chemistry. The T2(500) collapse is the most valuable single finding — a mechanistic negative result that converts "curriculum helps" into "curriculum is necessary here." The efficiency result deserves more attention than it receives. The task-construction recipe (200,000 verifiable tasks from 116 descriptors, plus structural textification) is reusable. Against this: curriculum learning and synthetic data scaling are established, so the novelty is the application rather than the principle. The frontier comparison carries much of the rhetorical weight but is confounded. And whether the models learned chemistry or learned SQM's biases remains unresolved — the question that determines whether any of this transfers to real campaigns. Language No editing needed. The writing is clean, terminology is defined at first use, and notation is consistent. The mixed polarity of the SQM sign convention is a presentation choice rather than a language problem, though it does force repeated clarification. Competing interests The author declares that they have no competing interests. Use of Artificial Intelligence (AI) The author declares that they used generative AI to come up with new ideas for their review.
Sneh Pankajbhai Vora· Zenodo (CERN European Organi...· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.
