Exploring ChatGPT-Assisted Qualitative Data: Comparing Human and AI-Assisted Coding Using Braun and Clarke’s Thematic Analysis

Abstract

The use of artificial intelligence (AI), particularly large language models such as ChatGPT, has significantly expanded across academia and research in recent years. Of growing interest is AI’s potential role in qualitative research, especially in expediting tasks such as literature review and synthesis, data management and coding, and manuscript preparation, all while striving to maintain the rigor expected of qualitative inquiry. This paper explores the degree of alignment between AI-generated thematic analysis and a traditional, manual approach to analyzing focus group data and potential uses of AI in qualitative research. Using Braun and Clarke’s thematic analysis framework, a publicly available focus group dataset was analyzed in three stages: (1) Data Familiarization, (2) Human-Led Thematic Analysis, and (3) AI-Generated Analysis using ChatGPT. The human coders followed all six phases of the framework prior to engaging with the AI, to prevent influence from AI-generated output. The comparison revealed partial overlap between the human and AI-generated categories, particularly at the level of broader categories and themes. However, ChatGPT’s analysis tended to reflect a more superficial reading of the data, lacking the interpretive depth and nuance demonstrated by the human coders. While AI tools like ChatGPT show potential for supporting certain aspects of qualitative analysis, this study highlights key limitations in their ability to fully replicate the complexity and contextual understanding that human-led analysis provides. Further exploration is needed to define appropriate and ethical roles for AI in qualitative research workflows.