Paper Info
Title
Learning Common Representation From RGB and Depth Images
Abstract
We propose a new deep learning architecture for the tasks of semantic segmentation and depth prediction from RGB-D images. We revise the state of art based on the RGB and depth feature fusion, where both modalities are assumed to be available at train and test time. We propose a new architecture where the feature fusion is replaced with a common deep representation. Combined with an encoder-decoder type of the network, the architecture can jointly learn models for semantic segmentation and depth estimation based on their common representation. This representation, inspired by multi-view learning, offers several important advantages, such as using one modality available at test time to reconstruct the missing modality. In the RGB-D case, this enables the cross-modality scenarios, such as using depth data for semantically segmentation and the RGB images for depth estimation. We demonstrate the effectiveness of the proposed network on two publicly available RGB-D datasets. The experimental results show that the proposed method works well in both semantic segmentation and depth estimation tasks.
Year
DOI
Venue
2018
10.1109/CVPRW.2019.00054
2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW)
Keywords
Field
DocType
semantic segmentation,depth prediction,RGB-D images,depth feature fusion,common deep representation,encoder-decoder type,depth estimation,common representation,multiview learning,missing modality,cross-modality scenarios,depth data,RGB-D datasets,deep learning architecture
Modalities,Network on,Feature fusion,Architecture,Pattern recognition,Segmentation,Computer science,RGB color model,Artificial intelligence,Deep learning,Machine learning
Journal
Volume
ISSN
ISBN
abs/1812.06873
2160-7508
978-1-7281-2507-7
Citations 
PageRank 
References 
0
0.34
13
Authors
2
Name
Order
Citations
PageRank
Giorgio Giannone100.34
Boris Chidlovskii241152.58