Paper Info
Title
Collaborative Learning for Hand and Object Reconstruction with Attention-guided Graph Convolution
Abstract
Estimating the pose and shape of hands and objects under interaction finds numerous applications including aug-mented and virtual reality. Existing approaches for hand and object reconstruction require explicitly defined physical constraints and known objects, which limits its application domains. Our algorithm is agnostic to object models, and it learns the physical rules governing hand-object interaction. This requires automatically inferring the shapes and physi-cal interaction of hands and (potentially unknown) objects. We seek to approach this challenging problem by proposing a collaborative learning strategy where two-branches of deep networks are learning from each other. Specifically, we transfer hand mesh information to the object branch and vice versa for the hand branch. The resulting optimi-sation (training) problem can be unstable, and we address this via two strategies: (i) attention-guided graph convo-lution which helps identify and focus on mutual occlusion and (ii) unsupervised associative loss which facilitates the transfer of information between the branches. Experiments using four widely-used benchmarks show that our frame-work achieves beyond state-of-the-art accuracy in 3D pose estimation, as well as recovers dense 3D hand and object shapes. Each technical component above contributes meaningfully in the ablation study.
Year
DOI
Venue
2022
10.1109/CVPR52688.2022.00171
IEEE Conference on Computer Vision and Pattern Recognition
Keywords
DocType
Volume
3D from single images, Deep learning architectures and techniques, Representation learning
Conference
2022
Issue
Citations 
PageRank 
1
0
0.34
References 
Authors
0
4
Name
Order
Citations
PageRank
Tze Ho Elden Tse100.34
Kwang In Kim2162578.90
Ales Leonardis31636147.33
Hyung Jin Chang430122.83