Skip to content

Misalignment Has a Personality: A Big Five Account of Emergent Misalignment

Jul 2026 · arXiv.org · Vol abs/2607.26389 · 0 citations · 35 references
Computer Science

TL;DR

Calibrated personality vectors transform an opaque safety phenomenon into a human-legible diagnostic profile by extracting activation directions for character traits from a single binary contrast, which can separate or steer behavior without establishing a calibrated scale.

Abstract

Fine-tuning a language model on data containing a narrow flaw, such as insecure code or incorrect mathematical answers, can cause broad misalignment through a mechanism that remains debated. We provide an interpretable account: in the models and corpora we study, misalignment behaves like a shift in personality. Prior work extracts activation directions for character traits from a single binary contrast, which can separate or steer behavior without establishing a calibrated scale. We instead extract personality vectors for the Big Five using a graded, three-level intervention and validate them on two open-weight models. The three levels are linearly ordered, with Cohen's d values of up to 6.2; the vectors transfer zero-shot and trait-specifically to an independent corpus; and their effects are strongest within a middle-layer band. Applied to training data, the vectors reveal that misaligned corpora across eight domains share a common Big Five signature: lower agreeableness and conscientiousness, together with higher extraversion and neuroticism. This signature is recovered by both models with a correlation of r = 0.94. Fine-tuning imprints the same profile, shifting the model's generations along the corresponding signature, with r = 0.83 using activation-based measurements and r = 0.90 using a text-based judge, while also shifting internal activations with r = 0.69. The same vectors characterize sycophancy as high extraversion and low conscientiousness rather than excess agreeableness, a distinction that a single direction cannot capture. Calibrated personality vectors transform an opaque safety phenomenon into a human-legible diagnostic profile.

View source

Similar papers

Jul 2026

What Models Express, Suppress, and Resist: Auditing Open-Weight LLMs with Persona Vectors

This work presents the first systematic application of persona vectors at this scale, compiling a 53-trait inventory across four behaviorally distinct domains and labeling every trait in two open-weight models as natural, steerable latent but amplifiable, or intractable (resistant to standard extraction).

W. Zeng, Ali Emami, Jinho D. Choi · 0 citations
Jul 2026

Persona Cartography: Charting Language Model Personality Traits in Weight Space

An unsupervised psychometric pipeline is introduced that recovers four interpretable behavioural factors from model rollouts that can be considered in terms of learning, scaling, and composing traits in weight space, providing a bridge between personality measurement, model editing, and safety.

L. Baines, Anton Gonzalvez Hawthorne, Mariia Koroliuk et al. · 1 citation
Review Jul 2026

Analyzing and Correcting Benevolence Bias in Large Language Models

Benevolence bias is identified and measure, a small but consistent tendency for aligned LLMs to lean toward the kinder, safer, more socially approved answer on value-laden survey questions, and is easy to diagnose and straightforward to fix.

Yuanzi Li, Jun-Hao Wang, Minghui Liu et al. · 0 citations
Review Aug 2026

Mind the Gaps: Mixture-of-Minds for Human Simulation

Anacreon is introduced, an audience simulation model that targets the individual level within a narrow, well-specified domain and reaches a state-of-the-art ordinal alignment of 0.775, the individual-level accuracy measure on which the field has converged, with a small residual bias.

P. Dahiya · 0 citations
Preprint Jul 2026

Transplanting, inverting, and preventing a misalignment persona: method-conditional emergent misalignment in Qwen2.5

Emergent misalignment (EM) --- the broad misbehaviour a language model acquires after fine-tuning on narrow harmful data --- is mediated in Qwen2.5 models by a latent persona direction, and that direction is causal in open weights. Transplanting it into a model that shares only pretraining with its source induces broad...

Lyndon Drake, Zandi Eberstadt · 0 citations
#natural language process... Preprint Aug 2026

Data Attribution of Emergent Misalignment with Persona Features

Emergent misalignment (EM) is the phenomenon where fine-tuning a language model on a narrow task leads to harmful behavior in unrelated domains. A leading mechanistic account attributes EM to persona features: latent directions acquired during pre-training that misaligned fine-tuning amplifies. We ask where these featu...

C. Vetter, David Kaczér, Lucie Flek et al. · 0 citations

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.