Preprint

DataRx: Missingness-Aware Sampling for Safer Large Language Model Task-Specific Fine-Tuning

Junbo Zhang Qianli Zhou Xinyang Deng Wen Jiang

Aug 2026 · 0 citations · 58 references

Computer Science

TL;DR

This paper proposes DataRx, a missingness-aware sampling method for selecting safety-critical examples based on the hypothesis that a safety sample is more effective when the selected examples provide safety signals that fill the missing parts of LLMs'safety capabilities.

Abstract

Task-specific fine-tuning can improve the performance of large language models (LLMs) on downstream tasks. However, our study reveals that task-specific fine-tuning can also weaken the safety guardrails of aligned LLMs. A widely adopted strategy for preserving safety during fine-tuning is to incorporate safety data. Although previous studies have shown that randomly mixing safety data can alleviate safety degradation, the underlying principle determining why some safety examples are more effective than others still remains unclear. In this paper, we propose DataRx, a missingness-aware sampling method for selecting safety-critical examples. DataRx is based on the hypothesis that a safety sample is more effective when the selected examples provide safety signals that fill the missing parts of LLMs'safety capabilities. DataRx's key insight is leveraging high-dimensional hidden representations rather than discrete tokens to quantify the safety signal gap between the target model's native response and the safety reference response. The results show that, with only 1% additional safety samples from BeaverTails, DataRx reduces the average attack success rate of Llama3-8B-Instruct across seven downstream tasks from 59.23% under random sampling to 13.70%. In addition, DataRx can be combined with the existing safety data synthesis method to further enhance safety defenses during fine-tuning. We hope that DataRx will inspire more data-centric defense research.

arXiv Semantic Scholar

View source

Similar papers

Preprint Jul 2026

DataShield: Uncovering Risky Fine-Tuning Data Across LLMs Through Consensus Subspace Alignment

DataShield is a data assessment framework that identifies risky fine-tuning samples and response segments through consensus subspace alignment over joint safety-critical semantic spaces derived from multiple safety-aligned LLMs, allowing both sample-level filtering and fine-grained segment-level masking.

Ze-Feng Wu, Weiwei Qi, Jielong Chen et al. · 4 citations

Book Open access Aug 2026

SAFT: Safety-Preserving Adaptation via Fine-Tuning Transfer for Large Language Models

SAFT (Safety-preserving Adaptation via Fine-tuning Transfer), a safety-preserving adaptation framework that decouples task learning from alignment preservation by learning a safety-guided task update on the paired pretrained base model, rectifying task gradients to avoid conflicting directions with respect to a safety objective, and then transferring the update to the frozen instruction model via parameter-space grafting.

Zhiwen Ruan, Yan Yang, Zhuocheng Liang et al. · Proceedings of the 32nd ACM... · 0 citations

Preprint Jul 2026

HyperSafe: Inference-Time Safety Recovery for Fine-Tuned Language Models

This work proposes HyperSafe, a framework that restores safety behavior by generating a model-specific Safe Side Network (SSN) for each fine-tuned checkpoint by using layer-wise activation fingerprints to capture how fine-tuning changes the model's inner representations.

Aznaur Aliev, Carlos Hinojosa, A. Eldesokey et al. · 0 citations

Preprint Aug 2026

HiRoute: Hierarchical Routed Prompt Tuning for Safety Alignment of Large Language Models

Experiments across three instruction-tuned models show that HiRoute achieves high safety rates across multiple safety benchmarks while preserving safe-response helpfulness, reducing over-refusal, and maintaining competitive performance on general-purpose tasks.

Fangzhou Chen, Shiji Zhao, Mengyan Wang et al. · 0 citations

Explaining Jailbreaks: Structured and Interpretable Safety Assessment for Large Language Models

This work proposes an explanation-aware safety framework that augments binary harmfulness detection with structured, human-interpretable explanations capturing severity, strategies, trigger spans, ratio-nales, and derived safety factors, and introduces a human–LLM hybrid annotation and canonicaliza-tion pipeline.

Sunghee Dong, Sungwon Yi, Kangmin Bae et al. · 0 citations

Preprint Aug 2026

Language-Specific Gaps in AI Safety Training Datasets

It is connected to a documented, persistent asymmetry in multilingual jailbreak robustness (single-turn attacks largely mitigated, multi-turn attacks still effective), arguing that this asymmetry is structurally consistent with where the audit finds training and evaluation data thinnest.

Chialuka Prisca-Mary Onuoha, Bright Etornam Sunu, Rashidat Sikiru · 0 citations