Name
Title | ||
|---|---|---|
On Hallucinating Context and Background Pixels from a Face Mask using Multi-scale GANs |
Abstract | ||
|---|---|---|
We propose a multi-scale GAN model to hallucinate realistic context (forehead, hair, neck, clothes) and background pixels automatically from a single input face mask, without any user supervision. Instead of swapping a face on to an existing picture, our model directly generates realistic context and background pixels based on the features of the provided face mask. Unlike facial inpainting algorithms, it can generate realistic hallucinations even for a large number of missing pixels. Our model is composed of a cascaded network of GAN blocks, each tasked with hallucination of missing pixels at a particular resolution while guiding the synthesis process of the next GAN block. The hallucinated full face image is made photo-realistic by using a combination of reconstruction, perceptual, adversarial and identity preserving losses at each block of the network. With a set of extensive experiments, we demonstrate the effectiveness of our model in hallucinating context and background pixels from face masks varying in facial pose, expression and lighting, collected from multiple datasets subject disjoint with our training data. We also compare our method with popular face inpainting and face swapping models in terms of visual quality, realism and identity preservation. Additionally, we analyze our cascaded pipeline and compare it with the progressive growing of GANs, and explore its usage as a data augmentation module for training CNNs. |
Year | DOI | Venue |
|---|---|---|
2018 | 10.1109/WACV45572.2020.9093568 | 2020 IEEE Winter Conference on Applications of Computer Vision (WACV) |
Keywords | DocType | Volume |
data augmentation module,face swapping models,visual quality,identity preservation,CNN,facial inpainting algorithms,hallucinated full face image,face image,GAN block,missing pixels,single input face mask,multiscale GAN model,background pixels,hallucinating context | Journal | abs/1811.07104 |
ISSN | ISBN | Citations |
2472-6737 | 978-1-7281-6554-7 | 0 |
PageRank | References | Authors |
0.34 | 16 | 4 |
Name | Order | Citations | PageRank |
|---|---|---|---|
| Sandipan Banerjee | 1 | 17 | 4.52 |
| Walter J. Scheirer | 2 | 773 | 52.81 |
| Kevin W. Bowyer | 3 | 11121 | 734.33 |
| Patrick J. Flynn | 4 | 4405 | 307.04 |