Preprint

From Specialization to Generalization: Instruction-tuned LLMs for Robust Harmful Content Mitigation

Lukas Edman Daryna Dementieva Alexander Fraser

Aug 2026 · 0 citations · 87 references

Computer Science

TL;DR

By thoroughly unifying 36 English hate speech datasets spanning multiple labeling schemes, this work fine-tune a generalist LLM, based on Qwen3 (Qwen Team, 2025), specifically for hate speech mitigation, demonstrating not only state-of-the-art performance on in-domain benchmarks but also substantial improvements in cross-domain and cross-lingual generalization--areas where encoder-based specialist classifiers often struggle.

Abstract

Large language models (LLMs) demonstrate impressive performance across a wide range of general NLP tasks; however, their effectiveness in sensitive domains, such as hate speech detection, remains less clear. Prior studies comparing prompted LLMs with state-of-the-art encoder-based models (e.g., BERT variants (Roy et al., 2023; D\"onmez et al., 2024)) have shown only marginal gains, suggesting that LLMs may not excel in hate speech detection or mitigation. In this work, we revisit this question through the lens of instruction tuning. By thoroughly unifying 36 English hate speech datasets spanning multiple labeling schemes, we fine-tune a generalist LLM, based on Qwen3 (Qwen Team, 2025), specifically for hate speech mitigation. Our results demonstrate not only state-of-the-art performance on in-domain benchmarks but also substantial improvements in cross-domain and cross-lingual generalization--areas where encoder-based specialist classifiers often struggle.

arXiv Semantic Scholar

View source

Similar papers

Preprint Jul 2026

Alignment Is All You Need: Instruction-Free Training for General Audio-Language Models

The results suggest that competitive MLLM can emerge from alignment alone, reducing multimodal extension to a lightweight projector-training problem that generalizes across modalities and adapts rapidly to each new LLM release.

Xuanru Zhou, Yiwen Shao, Jiahong Li et al. · 1 citation

Conference Open access Sep 2026

Gradient Enhancement Task Aware Post-training Quantization

This paper introduces Gradient Enhancement Task Aware Post-training Quantization, i.e., GTAQ, to address the generalization issue of Large Language Models, and extensively evaluates the LLaMA family of language models on WikiText, C4, and MMLU.

Yi-Hua Shao, Yang-Yang Gu, Min-Xi Yan et al. · Proceedings of the Thirty-Fi... · 1 citation

2026

Tiper: Tokenwise Prompt-Free Learning

Vision-language models (VLMs), such as contrastive language-image pre-training (CLIP), exhibit powerful zero-shot generalization capabilities. Parameter-efficient fine-tuning (PEFT) techniques, notably prompt learning, have been extensively explored to adapt these models to downstream tasks. However, their efficacy rem...

Tengfei Gong, Jun-Lin Wu, Yaxioong Chen et al. · IEEE Transactions on Geoscie... · 0 citations

Preprint Jul 2026

Data-Efficient Adaptation of LLMs via Attention Head Reweighting

Experiments on diverse open-source text classification datasets show that AHR can outperform standard baselines like LoRA when learning from limited samples, despite having 200-1000x fewer trainable parameters, as the authors' AHR only modifies ~0.0001% of the model's parameters.

Tuomas P. Oikarinen, Zixiao Chen, Charlotte Siska et al. · 0 citations

Preprint Aug 2026

Training-Free Token-Level Steering for LLM Personalized Co-Writing

This work introduces SteerWrite, a training-free framework designed for personalized co-writing that effectively adapts the base model to specialized domains without gradient updates, with specific designs tailored to small datasets.

Wenhao Mao, Chengbin Hou, Wei-Xiao Wang et al. · 0 citations

Preprint Jul 2026

The Effect of Multi-Lingual and Keyword Adversarial Injection on LLM Relevance Judgment

This work investigates the impact of cross-lingual prompt injection attacks on LLM-based relevance judgments using TREC Deep Learning collections and two open-weight models under established prompting frameworks, and demonstrates that multilingual query-based injections are highly effective in inflating relevance score...

Nguyen-Thanh-Thao Vo, Duy Duong Tuong, Oleg Zendel et al. · 0 citations