An integrated instance segmentation and deep feature fusion model for robust leaf species classification in natural environments

Abstract

The recognition of leaf segments and species in nature is a difficult process due to the changes in brightness, too much clutter in the background, and the overlapping of leaves due to which the accuracy falls down for 20-40% when using laboratory scans instead of field-ready applications. This research presents a unified framework of deep learning that includes Mask R-CNN -based instance segmentation and deep feature fusion in order to overcome this problem. The process of extracting a number of leaves took place through Detectron2 and the excessive stream extractor that included ResNet-50 and VGG-16 used in conjunction with other descriptive methods allowed providing a joint representation of 6,148 dimensions. The classifier was trained according to three methods: Baseline, PSO (particle swarm optimization), and Grid search. The masked fusion system cut off the noise in the background while retaining the important botanical features with the help of achieving the mean Intersection-over-Union equal to 0.752, detection precision of 98.39%, and mAP@0.5 of 0.8291. The ResNet-50 model enhanced with PSO techniques has the best performance among other technologies based on ResNet-50 (90% accuracy, F1 score 0.9002), while the technology of Grid search revealed a little higher result for the special techniques such as ResNet-50 plus traditional features (F1 score = 0.9135) which means that the efficiency of the methodology is contingent to the type of the features not the fact if the technology is better than the other. Comparative and ablation data analysis proved the efficiency of the instance segmentation used in combination with multi-mode fusion especially when used in occlusion-prone conditions. The system offers a scalable application for smart agriculture and biodiversity monitoring while being a good example of bridging the gap in automated environmental intelligence.