Desired-Update-Aligned Synthetic Data (DASA), which uses activation-gradient feedback from a frozen reference model to guide the optimization of continuous synthetic input embeddings, inspired by the role of activation gradients in local risk reduction, targets useful adaptation updates rather than source-text reconstruction or linguistic fluency.
Abstract
Is human readability necessary for effective fine-tuning of large language models? We investigate whether model-conditioned training representations can preserve or improve adaptation utility without requiring a human-readable textual form. We propose Desired-Update-Aligned Synthetic Data (DASA), which uses activation-gradient feedback from a frozen reference model to guide the optimization of continuous synthetic input embeddings. Inspired by the role of activation gradients in local risk reduction, DASA targets useful adaptation updates rather than source-text reconstruction or linguistic fluency. The resulting embeddings are used directly for downstream fine-tuning; discrete token projections are employed only for qualitative inspection. Experiments on six models from the Llama and Qwen families, ranging from 1B to 32B parameters, cover six benchmarks spanning knowledge, mathematical reasoning, code generation, and commonsense reasoning. Under matched LoRA adaptation settings, DASA achieves performance comparable to the source natural-language data and surpasses it in multiple configurations, while outperforming GRADMM in most comparisons. Further experiments cover general-domain and task-specialized source data. Under the evaluated synthesis settings, DASA provides a $3.6$--$4.9\times$ speedup over GRADMM with comparable peak GPU memory.
This work proposes a scalable reinforcement-learning-based approach for rewriting existing source texts into instances that are more difficult to translate for MT systems, using a large language model with Group Relative Policy Optimization (GRPO) and fine-tune a large language model with Group Relative Policy Optimiza...
Florian Zogaj, Jakob Hütteneder, Giovanni De Muri et al.· 1 citation
This work proposes augmenting existing benchmarks to increase translation difficulty by combining adversarial optimization with a differentiable translation difficulty estimator, and uses gradients from a combined difficulty and fluency objective to iteratively replace tokens in Adversarial Translation Optimization (AT...
William Kalikman, Šimon Sukup, Michal Tesnar et al.· European Association for Mac...· 2 citations
Corrective feedback is among the best-evidenced drivers of second-language acquisition, yet corrections delivered during lessons rarely accumulate into an actionable view of grammar mastery. Prompted frontier models can provide such a view from learner--tutor lesson transcripts, but they are costly at scale. We close t...
Marjan Celikik, A. Ramallo, Javier Morales· 0 citations
This work studies how to make LLMs natively generate TTS-friendly text, which is frame as a preference alignment problem: instead of relying on downstream rewriting modules, this work directly align LLMs to generate text optimized for spoken delivery.
Thibaut Thonet, Jos Rozen, Laurent Besacier· 0 citations
By thoroughly unifying 36 English hate speech datasets spanning multiple labeling schemes, this work fine-tune a generalist LLM, based on Qwen3 (Qwen Team, 2025), specifically for hate speech mitigation, demonstrating not only state-of-the-art performance on in-domain benchmarks but also substantial improvements in cro...
Lukas Edman, Daryna Dementieva, Alexander Fraser· 0 citations
The current robustness fine-tuning methods improve over standard fine-tuning and in-context learning, but the best-to-worst prompt gap remains as high as 40-57% of performance, and recent robustness-enhancing methods often fail to outperform the simplest data construction strategy: training on one template per batch.