Shadow AI in Swedish Health Care: Qualitative Analysis of Physicians’ Free-Text Answers
TL;DR
Shadow AI represents a renegotiation of professional boundaries, as physicians bypass institutional constraints to maintain professional efficacy and highlight a paradox in which the same tools that pose regulatory and safety risks also address real gaps in clinical and administrative support.
Abstract
Abstract Background The rapid emergence of artificial intelligence (AI) has outpaced its formal adoption in health care organizations, contributing to the emergence of Shadow AI, defined here as the use of unauthorized AI tools by medical professionals. Under the European Union Medical Device Regulation, AI tools used for clinical purposes must undergo conformity assessment before use; general-purpose tools such as ChatGPT have not done so, rendering their clinical application unauthorized at the regulatory level. While Shadow AI offers potential efficiency gains and higher performance, it poses significant risks to data privacy, clinical safety, and regulatory compliance. Despite its growing prevalence, empirical research on the purposes for which physicians use Shadow AI remains scarce. Objective This study explores the purposes for which physicians describe using Shadow AI in their work. Methods We conducted a cross-sectional survey of physicians employed in Swedish health care organizations (N=357; response rate~64%). Data were collected between December 2023 and January 2024 via a verified online panel. We conducted a qualitative content analysis of free-text responses on the use of unauthorized AI tools. We applied theoretical lenses from the sociology of professions and paradox theory to interpret the empirical findings. Results Physicians use Shadow AI for several purposes, which we grouped into 4 categories: clinical work and decision-making, administrative work, research and professional development, and technological interest and curiosity. More specifically, Shadow AI is used as a colleague and second opinion for clinical decision support (eg, differential diagnoses and rare cases), administrative tasks such as patient communication and documentation, and research aimed at staying up to date and exploring developments in generative AI. Physicians described using these tools compensated for perceived gaps in institutional systems, reducing workload, and accessing knowledge considered difficult to obtain through conventional channels. The findings reveal a tension between physicians’ drive to improve their practice and the regulatory and organizational constraints that render such use unauthorized. Conclusions Shadow AI used by physicians presents both opportunities and risks for health care professionals and organizations. Shadow AI indicates gaps where formal hospital systems may fail to meet health care professionals’ needs and signals a way for physicians to strengthen their experience-based knowledge. It represents a renegotiation of professional boundaries, as physicians bypass institutional constraints to maintain professional efficacy. The findings highlight a paradox in which the same tools that pose regulatory and safety risks also address real gaps in clinical and administrative support, suggesting that governance approaches must account for this tension rather than relying on prohibition alone.