Paper Info
Title
Occluded Apple Fruit Detection and Localization with a Frustum-Based Point-Cloud-Processing Approach for Robotic Harvesting
Abstract
Precise localization of occluded fruits is crucial and challenging for robotic harvesting in orchards. Occlusions from leaves, branches, and other fruits make the point cloud acquired from Red Green Blue Depth (RGBD) cameras incomplete. Moreover, an insufficient filling rate and noise on depth images of RGBD cameras usually happen in the shade from occlusions, leading to the distortion and fragmentation of the point cloud. These challenges bring difficulties to position locating and size estimation of fruit for robotic harvesting. In this paper, a novel 3D fruit localization method is proposed based on a deep learning segmentation network and a new frustum-based point-cloud-processing method. A one-stage deep learning segmentation network is presented to locate apple fruits on RGB images. With the outputs of masks and 2D bounding boxes, a 3D viewing frustum was constructed to estimate the depth of the fruit center. By the estimation of centroid coordinates, a position and size estimation approach is proposed for partially occluded fruits to determine the approaching pose for robotic grippers. Experiments in orchards were performed, and the results demonstrated the effectiveness of the proposed method. According to 300 testing samples, with the proposed method, the median error and mean error of fruits' locations can be reduced by 59% and 43%, compared to the conventional method. Furthermore, the approaching direction vectors can be correctly estimated.
Year
DOI
Venue
2022
10.3390/rs14030482
REMOTE SENSING
Keywords
DocType
Volume
agricultural robot, deep learning, fruit detection, point cloud, apple-harvesting robot, RGBD camera
Journal
14
Issue
Citations 
PageRank 
3
0
0.34
References 
Authors
0
5
Name
Order
Citations
PageRank
Tao Li138741.20
Qingchun Feng201.01
Quan Qiu300.34
Feng Xie400.34
J.-C. Zhao513552.42