On-policy distillation (OPD) is increasingly applied to frontier foundation model post-training. Prior work in this area has largely focused on algorithmic advances, yet it remains unclear how much OPD depends on its training questions and, in particular, how far this dependence can be reduced. Across two representativ...
Gengsheng Li, Mao Zheng, Ming-Yang Song et al.· 0 citations
Emotional expression can influence the safety decisions of large language models (LLMs), offering a potential avenue for improving safety alignment. Existing studies have mainly focused on how emotional expressions facilitate attacks under harmful requests, while overlooking their effects on benign requests. We find th...
Shu-Yi Miao, Yao-Jin Ma, Chen-Hang Cui et al.· 0 citations
Uncovering the internal mechanisms underlying the safety capabilities of large language models (LLMs) is crucial for developing trustworthy artificial intelligence. Currently, mechanistic interpretability studies on multilingual safety are largely confined to local components, such as isolated neurons. However, this st...
Shu-Yi Miao, Wang-Jie Qiu, Pengyang Shao et al.· 0 citations
ReflectRL is proposed, a lightweight plug-and-play framework that learns from Golden Negative Trajectories during on-policy training, and first uses these trajectories to elicit Reflective Reasoning, then applies Reflective-to-Direct Policy Transition to transfer the acquired reasoning behavior back to Direct Reasoning...
Jin-He Bi, Chennan Zhou, Zeng-Jie Jin et al.· 2 citations
This work proposes aligning cross-lingual thoughts and responses (ACTR), a framework that improves multilingual safety alignment by strengthening the use of existing safety reasoning, and devise neuron-selective consistency optimization (NSCO), which uses a frozen judge model to reward agreement between the safety cate...
Xian-Hui Zhang, Jian Yu, Cheng-Yu Xie et al.· 0 citations
On-policy self-distillation (OPSD) lets a language model learn from a frozen copy of itself that sees an answer or a worked solution. Giving the teacher this extra information seems to offer the student more to learn, but how much does it add beyond distillation itself? To isolate that contribution, we construct AMPLE-...
XiuYu Zhang, Wei Chow, Jun-Feng Fang et al.· 1 citation
SafeNexus is introduced, a cross-modal safety alignment framework that adopts a dedicated neuron-level intervention strategy that outperforms prevailing state-of-the-art approaches on safety benchmarks spanning diverse modality combinations, while effectively preserving utility.
Jian Yu, Fei Shen, Cong Wang et al.· arXiv.org· 0 citations
A neuron-level cross-dimensional safety alignment framework driven by modality- and language-shared safety neurons (MLS-Neurons) that significantly outperforms state-of-the-art approaches across diverse multilingual and multimodal safety benchmarks while preserving general utility.
Enyi Shi, Fei Shen, Chuancheng Shi et al.· arXiv.org· 0 citations
OPD-V is introduced, a visual OPSD paradigm that instantiates privileged information through the Positive Teacher and Negative Teacher that consistently improves reasoning performance while reducing training cost.
Anchored by this tri-axial framework, representative methods are systematically surveyed, the ongoing transition of continual learning is traced, and the key challenges, broader implications, and future directions arising from this paradigm shift are discussed.
Zhi-Yan Hou, Dan Zhang, Tao Feng et al.· 0 citations
This work presents Language Model Security Modules (LMSM), a security framework that adapts the separation behind Linux Security Modules (LSM) to LLM serving and gives advances in interpretability and model-internal analysis a common path to runtime enforcement.
XiuYu Zhang, Bo-Nan Ruan, Jun-Feng Fang et al.· 0 citations
Mechanist is an agentic system that uses AI as a scientific instrument for the autonomous discovery of mechanisms underlying AI intelligence, and develops a mechanism theory of belief, revealing how models represent world knowledge, form beliefs, infer the beliefs of others, and how these mechanisms emerge during pretr...
Mengru Wang, Jun-Feng Fang, Shuo-Fei Qiao et al.· 1 citation
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.
