Automated Tissue Detection For Enhancing Efficiency In Navigating Whole-Slide Images On A Gaming Device

Abstract

Introduction: Artificial intelligence (AI) and alternative input hardware have been proposed to optimize digital pathology workflows, yet their combined impact on slide navigation remains underexplored. Objectives: To determine whether automated tissue detection can improve review efficiency for a focused dermatopathology task on a handheld gaming device and characterize use patterns. Methods: We compared three approaches for detecting tissue sections on whole-slide images: Otsu–Watershed segmentation, zero-shot detection (OWLv2), and a fine-tuned transformer-based detector (DETR). Detections from the best-performing model were integrated into a custom viewer on a Steam Deck device for 9 dermatopathologists who subtyped basal cell carcinoma into four classes. The AI-assisted mode provided initial tissue centering, thumbnail cues, and “jump-to-tissue” controls. Results: DETR achieved higher mean F1 scores for tissue detection compared to Otsu–Watershed (0.96 (95% CI: 0.93-0.98) vs 0.84 (95% CI: 0.78-0.91)) and zero-shot OWLv2 (0.88 (95% CI: 0.82-0.94)), while all methods showed a median F1 of 1 across test images. AI assistance reduced time to review a case (p=0.005) and manual navigation burden (fewer joystick movements; p = <0.001). Interrater reliability for subtyping was substantial (Fleiss’ kappa 0.609). Conclusions: Integrating automated tissue detection with a controller-first interface on a handheld device improved navigation efficiency and maintained substantial agreement for a common dermatopathology task. These findings support targeted, human-in-the-loop AI as a practical means to streamline WSI navigation, particularly in portable or labeling-focused workflows.