This work introduces SteerWrite, a training-free framework designed for personalized co-writing that effectively adapts the base model to specialized domains without gradient updates, with specific designs tailored to small datasets.
Abstract
While Large Language Models (LLMs) show great promise for personalization, they often lack specialized domain knowledge. Conventional solutions like fine-tuning struggle with high computational costs and rapid data updates, while Retrieval-Augmented Generation fails to provide fine-grained, token-level steering. Furthermore, chat-based interfaces remain dominant, whereas productive co-writing paradigms have not yet been well exploited beyond the coding domain. To this end, we introduce SteerWrite, a training-free framework designed for personalized co-writing. Our method effectively adapts the base model to specialized domains without gradient updates, with specific designs tailored to small datasets. Experiments demonstrate that SteerWrite achieves state-of-the-art performance across diverse datasets, metrics, and models, significantly reducing human editing effort.
By thoroughly unifying 36 English hate speech datasets spanning multiple labeling schemes, this work fine-tune a generalist LLM, based on Qwen3 (Qwen Team, 2025), specifically for hate speech mitigation, demonstrating not only state-of-the-art performance on in-domain benchmarks but also substantial improvements in cross-domain and cross-lingual generalization--areas where encoder-based specialist classifiers often struggle.
Lukas Edman, Daryna Dementieva, Alexander Fraser· 0 citations
The results suggest that competitive MLLM can emerge from alignment alone, reducing multimodal extension to a lightweight projector-training problem that generalizes across modalities and adapts rapidly to each new LLM release.
Xuanru Zhou, Yiwen Shao, Jiahong Li et al.· 1 citation
GLA-LoRA establishes a unified learning strategy that synergistically integrates multi-granular contrastive learning with knowledge distillation and establishes that explicit global-local knowledge alignment is essential for achieving high-fidelity, parameter-efficient fine-tuning across diverse language tasks.
Despite their strong performance, large language models remain highly sensitive to prompt formulation. Prior work addresses this through refined data construction or through dedicated robustness objectives. We reproduce and compare these strategies under controlled conditions, and measure how effective they are in addressing models'prompt sensitivity. We find the current robustness fine-tuning methods improve over standard fine-tuning and in-context learning, but the best-to-worst prompt gap remains as high as 40-57% of performance. Moreover, the recent robustness-enhancing methods we test - CoIN for contrastive alignment and PPCL for consistency regularization - often fail to outperform the simplest data construction strategy: training on one template per batch. Our diagnostics explain these results. The auxiliary objectives move the quantity they penalize, but do not generalize beyond it. Additionally, data construction strategies differ due to the conflicting signs of per-template gradients on 57-64% of parameters. Thus, batches that mix formulations force the optimizer to reconcile competing updates instead of finding a shared, prompt-agnostic one.
This work proposes a distillation approach based on ranking supervision that consistently outperforms supervised fine-tuning as well as FKL and RKL baselines in Python code generation, multilingual generation, and data-science scenarios and offers guidance for future research in model compression.
Zhe Ding, Hui Ji, Su Pan et al.· Neural Networks· 0 citations
This one-day workshop provides hands-on, research-oriented training in fine-tuning large language models (LLMs) for extracting reliable, domain-specific insights from text data. Aimed at researchers across disciplines, it combines conceptual clarity with guided Jupyter notebooks, teaching participants how to fine-tune open-source models, apply efficient techniques like LoRA, and evaluate performance with reproducible workflows. No coding background is required — just curiosity and readiness to engage with code. By the end of the day, participants will leave with practical notebooks, evaluation pipelines, and a clear understanding of when and how to fine-tune (or when to rely on prompt engineering) for robust, ethical, and reproducible research.
Seraphine F. Maerz· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.
