Large-scale individual crown tree segmentation across entire white spruce forests using UAV hyperspectral imagery and deep learning

Abstract

Abstract. The development of high-performance, affordable UAVs has transformed vegetation monitoring, enabling observation of forest canopies at an unprecedented level of detail. UAV-derived datasets now provide high-fidelity structural and physiological information at the individual tree level across entire forest stands, offering novel insights into forest dynamics. In the context of increasing tree mortality, such data are becoming essential for understanding forest resilience and adaptation. However, exploiting this data requires effective individual tree crown segmentation algorithms (ITCS) at the forest scale, capable of tackling large-scale data and variability introduced by the environment. In this paper, we developed a new workflow designed to process UAV hyperspectral imagery at the forest scale, enabling automated ITCS and analysis. Our pipeline integrates hyperspectral-to-RGB conversion, ITCS, and centroid-based mask fusion. To assess the performance of our pipeline, we evaluated the model on two replicated white spruce common gardens in Canada, each comprising approximately 6,000 trees of similar age and structure. The experiments rely on a large multi-temporal dataset of hyperspectral imagery acquired during 60 UAV missions between 2022 and 2024, allowing us to evaluate the robustness of the proposed pipeline across a wide range of seasonal and acquisition conditions. Results show that the proposed pipeline achieves a mean segmentation performance of 0.536 mAP (0.885 mAP50) on the annotated dataset. At the forest scale, the system demonstrates strong detection capability with F1-scores of 0.948 at the Pintendre site and 0.863 at the Pickering site, successfully detecting most trees while maintaining stable performance across varying environmental conditions.