The AI Overtrust Paradox in Vulnerability Search: When Greater Trust in AI Reduces Perceived Search Effectiveness

Abstract

We investigate human overreliance on artificial intelligence (AI), a phenomenon in which users fail to critically verify AI outputs despite knowing the system is imperfect. To analyze this interaction, we model a cybersecurity scenario in which a human user and an AI chatbot collaborate to locate a vulnerable node in a large-scale distributed network. We consider a search process in which the probability distribution of the node’s location is updated using Bayes’ rule. In each round, the chatbot sequentially scans the nodes and suggests a potentially vulnerable node; the user then decides whether to accept the detection and end the search or refine the query and continue searching. The goal is to find an efficient scanning algorithm that maximizes the user’s expected satisfaction with the search results. Finally, we provide a formal proof of the “more for less” paradox, demonstrating that reducing trust in AI can, under certain conditions, increase perceived search effectiveness.