Jun 2026· arXiv.org· Vol abs/2606.26671· 0 citations· 38 references
Computer Science
TL;DR
This report provides a fully reproducible empirical post-training recipe for 8B-scale LLMs, and comprehensively dissects the capability trade-offs among instruction adherence, mathematical reasoning, code generation and general knowledge.
Abstract
Post-training alignment determines the reasoning and human preference following capabilities of large language models, yet most existing works withhold detailed data construction, filtering rules and training recipes, which hinders community reproducibility and lightweight model optimization. This work presents NebulaExp, a fully transparent, ablation-driven post-training pipeline built on Qwen3-8B-base, covering two orthogonal model branches: general instruct model and complex reasoning-specialized model. We curate a raw corpus of 3.84M multi-source SFT samples and a 200K verifiable RL candidate pool, and design an end-to-end data processing stack including response distillation, multi-dimensional cross-verification filtering, fine-grained difficulty grading, task classification and diversity-aware sampling. For the Instruct branch, our three-stage optimized supervised fine-tuning approach NebulaExp-Ins-SFT improves the average benchmark score from the 55.01 baseline of Qwen3-8B-nothink to 60.99. GRPO reinforcement learning then further elevates the average score to 61.85. For the Reasoning branch, medium-difficulty GRPO RL improves average reasoning score from 73.88 to 75.17. To address RL's dependency on task verifiers, we systematically investigate single-teacher and multi-teacher OPD (MOPD): utilizing merely 4K instruction-following samples and outperforms RL baseline by 3.26 points on IFEval with +4.43 average overall gain; MOPD fuses four domain-specialist teachers with merely 10K samples, lifting average performance by 4.18 over the base model. This report provides a fully reproducible empirical post-training recipe for 8B-scale LLMs, and comprehensively dissects the capability trade-offs among instruction adherence, mathematical reasoning, code generation and general knowledge.
Supervised fine-tuning is often discussed in terms of data volume and target quality, but the format of the supervision itself can change the reasoning strategy a model expresses in its outputs. We study this effect on a six-category reasoning benchmark spanning symbol transformation, text decryption, bit manipulation, gravitational constant estimation, numeral conversion, and unit conversion (drawn from the NVIDIA Nemotron Model Reasoning Challenge). Using NVIDIA Nemotron-3-Nano-30B-A3B with matched LoRA training settings, we compare three symbol-supervision formats: verbose English rule descriptions, compact family tags, and compact formula notation. We hypothesize that supervision renderings bias token-level reasoning priors, and that these priors transfer across task boundaries in multi-task SFT. In the canonical strict-rescore inventory, the best compact tag and formula checkpoints are statistically equivalent in aggregate within a pre-specified ±4-point margin: K8A-800 reaches 72.3% strict-scored overall accuracy and K8B-700 reaches 71.2% (TOST p = 0.003). Compact tags nevertheless provide a cleaner behavioral profile: an earlier K8A-400 checkpoint reaches 66.4% overall, 98.7% gravity accuracy, and 36.9% bit accuracy without the same contamination signatures. In contrast, verbose English rule descriptions are associated with heuristic parroting, with up to 57% of symbol failures at audited verbose checkpoints collapsing to a single remove-operator template, while formula notation is associated with cross-category contamination: numeric-looking predictions appear more often in text decryption (higher at five of six matched training steps under the canonical seed; matched-step means 15.8 vs. 11.7 numeric predictions per 157 text rows), and gravity failures at a representative K8B formula checkpoint shift toward shortcut stubs and explicit g = 9.8/9.81 fallbacks. We further show that checkpoint selection and strict evaluation auditing materially change branch decisions. Across three training seeds, neither compact format shows a consistent aggregate advantage, while the contamination signatures are partly seed-specific: the gravity-shortcut severity difference persists but is not exclusive to the formula branch, and the numeric–text signature does not reproduce under reseeding. These results support treating supervision format as a first-class hyperparameter for multi-task reasoning SFT in large language models—at least in this benchmark-and-model setting—rather than a mere rendering detail. Because such symbolic and procedural reasoning tasks recur in domains including cybersecurity, mathematics, and code generation, the same formatting choices plausibly shape the policy that any later reinforcement-learning stage would inherit, which we flag as future work.
Nhat Thanh Vu, M. Rashid, Fariza Sabrina· Electronics· 0 citations
As available training data approaches its physical limit, gains from Scaling Laws have begun to diminish. Consequently, improving Large Language Models (LLMs) now depends less on data expansion and more on higher-quality data utilization. However, in the context of large-scale corpora, existing refinement methodologies face significant limitations in quality, efficiency, and reliability: Rule-based approaches are constrained by fixed heuristics and struggle with instance-level variations; LLM-based approaches improve quality but fail to meet the efficiency and reliability requirements of large-scale data processing. To address these challenges, we propose UltraX, a function-calling refinement framework for large-scale pre-training data that completes the editing function space by introducing insertion in addition to deletion and modification, enabling fine-grained instance-level editing. Specifically, UltraX builds a reliable program-supervision generation pipeline. In this pipeline, dataset-adaptive prompt optimization first guides an expert LLM to produce high-quality end-to-end refined texts, and Line Alignment Mapping and Dynamic Context Replacement then convert original-refined text pairs into structured program supervision. Meanwhile, UltraX improves supervision quality and stabilizes the training distribution with low-confidence example filtering and ratio-controlled sampling by operation combination. During inference and execution, it normalizes and validates model outputs through sliding-window prediction, global operation aggregation, and systematic post-processing, improving the stability and reliability of large-scale execution. Experiments show that UltraX achieves the highest average performance across all corpora and also matches or surpasses baselines with fewer training tokens, demonstrating stronger data efficiency and refinement reliability.
Xinlong Zhao, Dongsheng Liu, Hengyu Zhao et al.· 0 citations
Test-time adaptation (TTA) has emerged as a popular paradigm for improving the performance of vision-language models (e.g., CLIP) on downstream tasks. Among existing CLIP-based TTA methods, Test-Time Prompt Tuning (TPT) is a pioneering work that optimizes textual prompts using multiple test-time augmentations and remains a strong baseline to date. In this work, we revisit TPT and reveal that its optimization can be interpreted as implicitly learning from self-generated pseudo labels. Building on this perspective, we propose a unified self-ensembling framework (USE) that ensures consistency between the optimization and inference stages. During optimization, we introduce a simple yet effective self-ensembling (SE) strategy that emphasizes the test image itself over its augmented views adaptively to obtain more reliable pseudo labels. To fully exploit the potential of augmentations, we further apply the same strategy at inference time, unifying the objectives of both stages. Notably, SE can also act as a lightweight optimization-free TTA method. Extensive experiments across multiple datasets demonstrate that SE and USE outperform their counterparts, respectively. Furthermore, SE yields consistent performance gains when integrated with existing TTA methods. The code is available at https://github.com/sirujiang/USE.
Siru Jiang, Jian Liang, Ran He et al.· 0 citations
Enterprise question answering requires models to acquire proprietary knowledge without discarding general capabilities. We present Wnuan, a three-stage pipeline that constructs task-oriented supervision from documents, performs supervised fine-tuning with general-data replay, and applies reinforcement learning to residual errors. On the 707-question WnuanBench, the primary 32B route raises acceptable-answer rate (AAR) from 52.76% before adaptation to 80.06% after SFT and 91.51% after RL. Under a matched 100-update protocol, residual-error sampling outperforms full-pool and size-matched random sampling by 3.11 and 2.97 points, respectively. Source-cluster bootstrap intervals remain above zero for both contrasts, and a same-domain validation set preserves the ordering. The general-benchmark average decreases by 5.17 points across the route, concentrated in instruction following. The automatic evaluation ensemble agrees with an authoritative domain expert on 90.5% of a stratified Wnuan-Inst response sample. These results characterize both the gains and the general-capability cost of staged enterprise adaptation.
Xiaofeng Shi, Xiaosong Qiu, Wenxin Ma et al.· 0 citations
Not all training samples contribute equally to large language model fine-tuning. Selecting informative training samples can reduce the computational cost while preserving downstream performance. Many existing data selection methods rely on indirect heuristics, such as data quality, diversity or reasoning trace length. However, the effectiveness of these fixed criteria is task-dependent and difficult to generalize across diverse downstream tasks. Perplexity-based data selection provides a simple and model-aware solution to estimate the sample difficulty, but existing approaches typically score the entire training sequence and ignore the difference in learning objectives of language modeling and reasoning tasks. In this paper, we propose PPL-Factory, a simple and interpretable data selection framework that combines task-aware perplexity-based scores and data budget-aware selection criteria. Experiments on GSM8K demonstrate that PPL-Factory outperforms other state-of-the-art data selection methods using only $1\%$ of the training set. With $10\%$ of the data, PPL-Factory exceeds full-data fine-tuning accuracy by 0.9 on GSM8K and 4.8 on MATH. Overall, our results demonstrate that task-aware and budget-aware perplexity-based selection provides an effective and applicable approach for efficient fine-tuning.
The meteoric rise of large language models (LLMs) has reshaped natural language processing, inspiring new approaches to data integration tasks such as entity matching (EM). While proprietary models like GPT-4 deliver strong performance, open-source alternatives (e.g., Mistral, DeciLM, Gemma-3) require supervised fine-tuning (SFT) to excel in specialized domains. However, naïvely training billion-parameter scale LLMs on uncurated corpora is computationally prohibitive and environmentally costly, often taking days while introducing inefficiencies that limit rapid application deployment. We introduce AssistEM, a framework for efficient LLM adaptation to EM via principled data selection. AssistEM integrates NER and TF-IDF signals into a composite score, filters entity pairs with domain-level thresholds, constructs denoised semantic embeddings, and applies graph-based clustering with class-aware sampling to obtain a compact yet representative set of ∼\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\sim $$\end{document}8000 high-quality training instances. This yields rapid specialization: AssistEM-trained LLMs converge in under 4.5 hours–versus 12+ hours for DeciLM-7B and 24+ hours for Yi-34B–while surpassing GPT-4-0613 by 4.3 F1 points and Ditto by 11 F1 points across diverse EM benchmarks. By aligning data quality with model specialization, AssistEM demonstrates that selective fine-tuning not only accelerates adaptation but also improves training efficiency (requiring fewer GPU hours), enabling open-source LLMs to rival–and in some cases outperform–closed-source models. These results highlight data selection as a critical lever for sustainable, domain-specific LLM deployment.
John Bosco Mugeni, S. Lynden, Toshiyuki Amagasa et al.· International Journal of Dat...· 0 citations