Title | ||
---|---|---|
Neural Emotion Director: Speech-preserving semantic control of facial expressions in “in-the-wild” videos |
Abstract | ||
---|---|---|
In this paper, we introduce a novel deep learning method for photo-realistic manipulation of the emotional state of actors in “in-the-wild” videos. The proposed method is based on a parametric 3D face representation of the actor in the input scene that offers a reliable disentanglement of the facial identity from the head pose and facial expressions. It then uses a novel deep domain translation framework that alters the facial expressions in a consistent and plausible manner, taking into account their dynamics. Finally, the altered facial expressions are used to photo-realistically manipulate the facial region in the input scene based on an especially-designed neural face renderer. To the best of our knowledge, our method is the first to be capable of controlling the actor's facial expressions by even using as a sole input the semantic labels of the manipulated emotions, while at the same time preserving the speech-related lip movements. We conduct extensive qualitative and quantitative evaluations and comparisons, which demonstrate the effectiveness of our approach and the especially promising results that we obtain. Our method opens a plethora of new possibilities for useful applications of neural rendering technologies, ranging from movie post-production and video games to photo-realistic affective avatars. |
Year | DOI | Venue |
---|---|---|
2022 | 10.1109/CVPR52688.2022.01822 | IEEE Conference on Computer Vision and Pattern Recognition |
Keywords | DocType | Volume |
Image and video synthesis and generation, Face and gestures, Vision + graphics | Conference | 2022 |
Issue | Citations | PageRank |
1 | 0 | 0.34 |
References | Authors | |
0 | 4 |
Name | Order | Citations | PageRank |
---|---|---|---|
Foivos Paraperas Papantoniou | 1 | 0 | 0.34 |
Panagiotis Paraskevas Filntisis | 2 | 4 | 3.92 |
Petros Maragos | 3 | 3733 | 591.97 |
Anastasios Roussos | 4 | 288 | 15.83 |