Human discernment of artificial intelligence in online markets can be shaped by experience and training

Abstract

Artificial intelligence systems can produce information that closely resembles real and human-created information, making it difficult to accurately distinguish between synthetic (AI-generated) and non-synthetic content. We explored whether individual differences in attitudes toward AI or one’s history of engagement with AI were related to individual differences in the ability to distinguish AI-generated from real/human-created content. Though attitudes did not predict performance on a human/AI discernment task, more frequent and varied engagement with AI platforms was associated with significantly lower discernment scores. We next tested if we could improve human evaluators’ discernment accuracy through targeted intervention. Participants were assigned to either an experimental group ( n = 60) or a control group ( n = 57). The experimental group completed an intervention in which they first trained with accurately labeled human/AI exemplars and then completed a series of discernment trials while being given immediate feedback and a monetary incentive for each accurate choice. The control group participated in similar exercises but trained with unlabeled human/AI exemplars and received delayed feedback and monetary incentivization based on their performance. While there were no group differences in discernment accuracy at a baseline assessment, participants in the experimental group demonstrated significant gains in discernment accuracy (relative to baseline) at both an intermediate and final assessment, while the control group showed no improvement across assessments. These findings provide evidence that frequent interactions with AI platforms can diminish sensitivity to the differences between real and synthetic content, but that targeted interventions can meaningfully benefit human evaluators' discernment skill.