Name
Abstract | ||
|---|---|---|
We address the problem of recovering the 3D geometry of a human face from a set of facial images in multiple views. While recent studies have shown impressive progress in 3D Morphable Model (3DMM) based facial reconstruction, the settings are mostly restricted to a single view There is an inherent drawback in the single-view setting: the lack of reliable 3D constraints can cause unresolvable ambiguities. We in this paper explore 3DMM-based shape recovery in a different setting, where a set of multi-view facial images are given as input. A novel approach is proposed to regress 3DMM parameters from multi-view inputs with an end-toend trainable Convolutional Neural Network (CNN). Multiview geometric constraints are incorporated into the network by establishing dense correspondences between different views leveraging a novel self-supervised view alignment loss. The main ingredient of the view alignment loss is a differentiable dense optical flow estimator that can back-propagate the alignment errors between an input view and a synthetic rendering from another input view, which is projected to the target view through the 3D shape to be inferred. Through minimizing the view alignment loss, better 3D shapes can be recovered such that the synthetic projections from one view to another can better align with the observed image. Extensive experiments demonstrate the superiority of the proposed method over other 3DMM methods. |
Year | DOI | Venue |
|---|---|---|
2019 | 10.1109/CVPR.2019.00105 | 2019 IEEE/CVF CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION (CVPR 2019) |
Field | DocType | Volume |
Regression,Pattern recognition,3d geometry,Computer science,Convolutional neural network,3d shapes,Differentiable function,Artificial intelligence,Rendering (computer graphics),Optical flow,Estimator | Journal | abs/1904.04473 |
ISSN | Citations | PageRank |
1063-6919 | 2 | 0.36 |
References | Authors | |
0 | 8 | |
Name | Order | Citations | PageRank |
|---|---|---|---|
| Fanzi Wu | 1 | 5 | 1.44 |
| Linchao Bao | 2 | 194 | 9.77 |
| Yajing Chen | 3 | 2 | 0.36 |
| Yonggen Ling | 4 | 16 | 3.63 |
| Yibing Song | 5 | 116 | 8.95 |
| Songnan Li | 6 | 323 | 19.67 |
| King Ngi Ngan | 7 | 2383 | 185.21 |
| Wei Liu | 8 | 4041 | 204.19 |