Comparison of the effectiveness of selected tools for detecting texts generated by artificial intelligence

Abstract

This paper compares the effectiveness of selected publicly available tools for detecting AI-generated text: GPTZero AI Detector, ZeroGPT AI Detector, Copyleaks AI Detector, and Scribbr AI Detector. The experiment included 150 text samples in Polish and English, divided into human-written texts, AI-generated texts, and texts paraphrased using AI. ZeroGPT achieved the highest overall accuracy, equal to 64.0%, and the highest F1-score, equal to 77.7%. GPTZero achieved an accuracy of 58.7%, while Scribbr and Copyleaks achieved lower results, equal to 44.0% and 42.7%, respectively. Fully AI-generated texts were detected more effectively than AI-paraphrased texts. The overall accuracy was similar for Polish and English samples, equal to 52.7% and 52.0%, respectively.