The Adaptation Dilemma: Cultural Fit Does Not Guarantee Safety in Mental-Health LLMs

Abstract

People increasingly take mental-health concerns to general-purpose AI chatbots. A first wave of safety research has begun to map LLM failure points in this context. When that work considers culture, it usually reads the problem as a lack of fit: if the model does not match the user, the fix is to make it match more closely. We argue that this perspective is incomplete, because cultural fit and safety are not the same thing. By safety we mean the absence of harm caused by the interaction itself. We sort these harms into two classes: imposition, where the model forces its default on a user it does not fit, and collusion, where it adapts to a framing that keeps the user unwell. Because a response that fits a user’s own framing is more credible to them, moving a model toward that framing lowers the risk of imposition while raising both the likelihood of collusion and the damage collusion does when it occurs, so improved cultural sensitivity trades one exposure for another rather than reducing exposure overall. We argue that the reinforcement learning layer increasingly pushes these models toward collusion, while the safeguards in current use are poorly suited to detecting that failure or to limiting the harm it does. We propose a taxonomy of six culture-related failure modes, three of imposition and three of collusion. We argue that evaluating safety requires an anchor outside the user’s in-the-moment approval: responses must be assessed against the harms and benefits the interaction produces for the user, judged both by an external standard and against the goals the user has stated for their care.