Paper Info
Title
HifiHead: One-Shot High Fidelity Neural Head Synthesis with 3D Control.
Abstract
We propose HifiHead, a high fidelity neural talking head synthesis method, which can well preserve the source image's appearance and control the motion (e.g., pose, expression, gaze) flexibly with 3D morphable face models (3DMMs) parameters derived from a driving image or indicated by users. Existing head synthesis works mainly focus on low-resolution inputs. Instead, we exploit the powerful generative prior embedded in StyleGAN to achieve high-quality head synthesis and editing. Specifically, we first extract the source image's appearance and driving image's motion to construct 3D face descriptors, which are employed as latent style codes for the generator. Meanwhile, hierarchical representations are extracted from the source and rendered 3D images respectively to provide faithful appearance and shape guidance. Considering the appearance representations need high-resolution flow fields for spatial transform, we propose a coarse-to-fine style-based generator consisting of a series of feature alignment and refinement (FAR) blocks. Each FAR block updates the dense flow fields and refines RGB outputs simultaneously for efficiency. Extensive experiments show that our method blends source appearance and target motion more accurately along with more photo-realistic results than previous state-of-the-art approaches.
Year
DOI
Venue
2022
10.24963/ijcai.2022/244
International Joint Conference on Artificial Intelligence
Keywords
DocType
Citations 
Computer Vision: Neural generative models, auto encoders, GANs,Computer Vision: 3D Computer Vision,Computer Vision: Applications
Conference
0
PageRank 
References 
Authors
0.34
0
7
Name
Order
Citations
PageRank
Feida Zhu100.68
Junwei Zhu222.14
Wenqing Chu3466.74
Ying Tai421325.74
Zhifeng Xie501.01
Xiaoming Huang601.35
Chengjie Wang74319.03