First-in-human evaluation of real-time pixel-level AI-assisted anatomical segmentation in neurosurgery: pituitary surgery as an exemplar

Abstract

Precise anatomical navigation is fundamental to safe endoscopic pituitary surgery, a high-stakes procedure characterised by a challenging learning curve. While traditional navigation systems often rely on workflow-disrupting probes or static preoperative imaging, advancements in computer vision AI (CVAI) now enable dynamic, real-time pixel-level anatomical segmentation directly from live surgical video. Our group has previously conducted a series of preclinical human-computer interaction studies to refine the system’s design, alongside digital and high-fidelity physical simulations demonstrating the potential benefit of AI assistance in improving surgical performance, training, and safety. Building on this foundation, the current study represents a first-in-human evaluation of real-time pixel-level CVAI anatomical segmentation in the neurosurgical operating room - assessing feasibility, human factors and clinical outcomes, while iteratively improving the system. Guided by the DECIDE-AI and IDEAL frameworks, this single-centre evaluation comprises an initial proof-of-concept of CVAI anatomical segmentation in endoscopic transsphenoidal pituitary surgery. The AI model utilised a DINOv3-derived vision transformer architecture, deployed via a high-performance edge computing unit to achieve low-latency real-time inference without reliance on cloud infrastructure. Feasibility and functionality were assessed via structured questionnaire, prospective observation, and blinded retrospective review of the recordings of the endoscopic surgical video feed and wider operating room environment. Continuous multi-stakeholder feedback through validated human factors surveys drove iterative technical refinements between cases. Routine clinical outcomes, aligning with the standard pituitary surgery core outcome set, were collected. Eight patients with pituitary adenomas were enrolled. The CVAI system was successfully deployed in six cases, demonstrating acceptable real-time pixel-level sella segmentation accuracy. Deployment failed pre-operatively in two cases owing to a single platform-level boot-configuration issue. Iterative refinement between cases was driven by our experience and surgical team feedback. This resulted in the integration of additional anatomical structure segmentations (e.g., carotid arteries), enhanced model accuracy via training dataset expansion, and hardware firmware upgrades. Multi-stakeholder surveys demonstrated satisfactory system feasibility, usability, and acceptability among the surgical team. Both prospective observation and retrospective video review confirmed the absence of adverse events, including no significant distraction to the primary surgeon, and there were no AI-related clinical complications. This first-in-human early clinical evaluation (IDEAL Stage 1) of real-time pixel-level AI anatomical segmentation in live neurosurgery demonstrates feasibility, showcases iterative system evolution, and reports clinical and human factors outcomes. Future work will include a larger single-centre case series (IDEAL Stage 2a) with more surgical teams to further iterate the system and explore its impact on safety, training and workflow. As the underpinning AI models improve and integrate with other intra-operative navigational technologies, such tools will likely be the cornerstone of intra-operative surgical decision support systems.