Combining probabilistic features and semantic features for AI-Generated text detection

Abstract

The proliferation of Large Language Models (LLMs) such as ChatGPT and Gemini has resulted in a surge of AI-generated text across various domains. However, the widespread use of this technology raises concerns regarding the generation of misinformation and malicious content. To address this challenge, we propose a novel AI-generated Text Detection model combining Probabilistic and Semantic features (ATDPS). Our model extracts semantic features using a pre-trained language model and combines them with probabilistic features generated by multiple LLMs. A temporal convolutional network is employed to process sequence probabilistic features, effectively capturing temporal characteristics within the text. To ensure data coherence and diversity, our dataset includes text generated by a variety of LLMs, including the latest models like GPT-4. Experimental results demonstrate ATDPS's superior performance over existing baselines in terms of accuracy, precision, recall and F1 score, highlighting its potential and effectiveness in detecting AI-generated text.