Geautomatiseerde objectmanipulatie in hardfruitboomgaarden – Robotisch snoeien en oogsten

Abstract

Growing pome fruit requires knowledge of the tree and its circumstances. Throughout the year, experienced cultivators manage the orchard in various ways, including certain manipulations in the tree such as pruning, thinning and harvesting. Nowadays, these tasks are still mainly performed manually. An extensive activity survey combined with a literature and market study, performed at the start of this research, pointed out the urge of introducing automation and robotics in pome fruit cultivation, especially for the most labour-intensive orchard management tasks of harvesting and pruning. This research took the first steps towards robotic pruning by contributing on four aspects of the robotic application: the development of a system to assess the branch structure of a tree; the creation of an elaborated dataset of pruning data; the investigation of pruning rules for robotic pruning decision-making; and the development of a multi-functional robotic orchard platform to perform robotic manipulations of various kinds. Multiple branch assessment approaches were investigated during this research. Firstly, a line model approach was implemented to represent the branch structure as a hierarchical graph of line segments. This method was able to identify the larger components of the tree such as the trunk and the main branches. However, due to interference with noise, smaller branches and details of the tree could not be determined adequately by this approach. Furthermore, a cylindrical model approach was investigated as well. Using cylindrical elements to represent branch parts, a similar hierarchical structure was built up. Besides trunk and main branches, this method was able to identify one level of branches further in a detailed way, but facing with difficulties induced by the noisy data as well. Finally, some recent developments in literature were studied with the aim of adopting (parts of) these techniques into the self-developed methods of this research. During this research, a dataset of both point cloud and RGB-data in winter dormancy stage of the trees has been established. With over 3,200 tree scans, this dataset grew up to one of the largest of this kind in the world. Moreover, this dataset has many assets for other research related to fruit cultivation, as it will be published and publically available. The data was captured during various weather and lighting conditions, and at multiple moments throughout seasons and years. All these aspects ensure that this dataset is suitable for training robust neural networks, as well as for tree growing assessment over time. As objective and digitally implementable pruning rules did not exist yet, this research aimed to determine such rules for robotic pruning decision-making. In a first research track, sets of simplified deterministic pruning rules were defined in cooperation with professional pruners. These rules were applied on some test objects to evaluate the consequences on growth and yield for the upcoming years. On the other hand, the data of the above-mentioned dataset was used to identify the current manual pruning behaviour. Via the developed comparison algorithm, the pruning actions were digitalised with a F1-score of 82.2 %. In future work, these digital pruning points will be used as labelled data for developing an AI-based prediction model for new pruning events. Within this research, a robotic setup has been developed with the objective of executing robotic manipulations in orchard environments. This setup was created in two phases, ending up with an 8 DoF manipulator mounted on a mobile platform. For each phase in the development, the hardware setup was validated during field tests. The conversion from the first to the second version led to an increase of success rate of 27 % and an improvement of the cycle time of 28 %, resulting in 92 % of average success at a speed of 8.9 s/cut. The setup was tested for multiple tree architectures, performing robustly and consistently in all tested systems. Based on an extensive techno-economic study, it became clear that such a robotic manipulator setup for fruit cultivation, as developed in this research, could only become profitable if it would be employed as much as possible during the cultivation season. Therefore, integration of other functionalities within the orchard was investigated as well. Besides the main use case of pruning, the integration of a harvesting unit was established in this research as a Proof-of-Concept. The harvesting use case has been preliminarily tested, resulting in a success rate of 68 % and 76 % for respectively picking pear and apple. Combining both the pruning and the harvesting use case in a multi-functional robotic orchard platform, the techno-economic study concluded that such system could become profitable with a plausible break-even point of 8.4 years.