Paper Info
Title
Panoptic 3d Mapping And Object Pose Estimation Using Adaptively Weighted Semantic Information
Abstract
We present a system capable of reconstructing highly detailed object-level models and estimating the 6D pose of objects by means of an RGB-D camera. In this work, we integrate deep-learning-based semantic segmentation, instance segmentation, and 6D object pose estimation into a state of the art RGB-D mapping system. We leverage the pipeline of ElasticFusion as a backbone and propose modifications of the registration cost function to make full use of the semantic class labels in the process. The proposed objective function features tunable weights for the depth, appearance, and semantic information channels, which are learned from data. A fast semantic segmentation and registration weight prediction convolutional neural network (Fast-RGBD-SSWP) suited to efficient computation is introduced. In addition, our approach explores performing 6D object pose estimation from multiple viewpoints supported by the high-quality reconstruction system. The developed method has been verified through experimental validation on the YCB-Video dataset and a dataset of warehouse objects. Our results confirm that the proposed system performs favorably in terms of surface reconstruction, segmentation quality, and accurate object pose estimation in comparison to other state-of-the-art systems. Our code and video are available at https://sites.google.com/view/panoptic-mope.
Year
DOI
Venue
2020
10.1109/LRA.2020.2970682
IEEE ROBOTICS AND AUTOMATION LETTERS
Keywords
DocType
Volume
RGB-D perception, object detection, segmen-tation and categorization, mapping
Journal
5
Issue
ISSN
Citations 
2
2377-3766
0
PageRank 
References 
Authors
0.34
0
3
Name
Order
Citations
PageRank
Dinh-Cuong Hoang100.34
Achim J. Lilienthal21468113.18
Todor Stoyanov326026.07