AI Alignment Training Paradigms and Cross-Model Differences in AI Persuasive Behavior

Abstract

Reinforcement learning from human feedback (RLHF) and constitutional artificial intelligence (CAI) are two ways to shape model behavior. Among them, CAI is built around avoiding different categories of harm, including racism, sexism, toxic, dangerous and illegal content. This article integrates the existing empirical research on the persuasiveness of large language models, mainly focusing on real-time interactive experiments and computational text analysis, and studies the training mechanism behind the two paradigms. As a result, the article predicts that compared with RLHF, CAI only suppresses the strategies within the injury categories it lists, so that it will not be more persuasive than the model trained by RLHF. Then, this paper compares this prediction with the recent large-scale cross-model persuasion benchmark. The results show that the model trained by CAI is the most convincing among the models tested. Therefore, this paper puts forward three candidate mechanisms to guide future research. First, the revision of the training stage may improve the quality of the argument. Second, the non-evasive interaction style may increase the credibility of perception. Third, sycophancy under counter pressure may undermine RLHF-trained systems in confrontational settings. At present, the difference in alignment paradigm cannot be a reliable proxy to reduce the risk of artificial intelligence persuasion.