Deep Learning-Based Segmentation, Action, and Phase Recognition in Laparoscopic Cholecystectomy: A Review

Abstract

The integration of artificial intelligence into biomedical image analysis represents a paradigm shift in clinical decision support and surgical informatics. Laparoscopic cholecystectomy (LC) is a widely performed minimally invasive procedure but remains technically challenging due to limited visibility, anatomical variations, and the risk of complications such as bile duct injury. Artificial intelligence (AI), particularly deep learning (DL), has shown great potential for improving intraoperative understanding through automated surgical video analysis. This review summarizes recent advances in DL-based LC video analysis, focusing on semantic segmentation and surgical action or phase recognition. Existing approaches are categorized into convolutional neural networks (CNNs), transformer-based architectures, and hybrid spatio-temporal models. For segmentation, U-Net variants, DeepLab architectures, and vision transformers are reviewed for identifying anatomical structures and surgical instruments. For action and phase recognition, CNN-LSTM frameworks, 3D CNNs, and video transformers are discussed for modeling surgical workflows and temporal dependencies. We also summarize commonly used datasets, including Cholec80 and CholecSeg8k, and evaluation metrics such as Dice score, IoU, accuracy, and F1-score. Major challenges include data scarcity, inter-surgeon variability, domain shift, and real-time computational constraints. Future research should emphasize self-supervised learning, multi-modal fusion, and clinically explainable AI to improve robustness, generalizability, and real-world surgical deployment.