Name
Abstract | ||
|---|---|---|
The image-to-image translation task has made significant progress by relying on conditional generative adversarial networks. However, for many tasks, multiple condition images are required. This paper considers a very classic application scenario, using lines and appearance to synthesize photo-realistic images, describing structure and appearance information, respectively, for example, generating realistic face images from portrait drawings and color scribbles, and generating photos from sketches and texture patches. The key to this type of task is how to fuse the two conditional information. We propose an image translation system driven by line and appearance images, introducing a modular architecture for condition fusion. Unlike the previous condition fusion schemes, its main body of the generator is composed of stacked modulation units (MUs). Here, structural features and appearance features are progressively incorporated via cascaded MUs, each of which pays attention to the local regions. The visualization experiment shows that such a scheme lets the network automatically learn to decompose the fusion process as multiple sub-steps in latent spaces. Our model produces higher quality results quantitatively and qualitatively compared to the state-of-the-art method on different tasks and datasets. The ablation study demonstrates the effectiveness of the MUs and intuitively explains the process of feature fusion through visualization. |
Year | DOI | Venue |
|---|---|---|
2022 | 10.1016/j.neucom.2022.06.007 | Neurocomputing |
Keywords | DocType | Volume |
Image Synthesis,Image-to-Image Translation,Feature Fusion,Generative Adversarial Networks | Journal | 503 |
ISSN | Citations | PageRank |
0925-2312 | 0 | 0.34 |
References | Authors | |
0 | 3 | |
Name | Order | Citations | PageRank |
|---|---|---|---|
| Wuyang Luo | 1 | 0 | 0.68 |
| Su Yang | 2 | 110 | 14.58 |
| Weishan Zhang | 3 | 396 | 52.57 |