Title | ||
---|---|---|
Learning Hierarchical Semantic Image Manipulation through Structured Representations. |
Abstract | ||
---|---|---|
Understanding, reasoning, and manipulating semantic concepts of images have been a fundamental research problem for decades. Previous work mainly focused on direct manipulation on natural image manifold through color strokes, key-points, textures, and holes-to-fill. In this work, we present a novel hierarchical framework for semantic image manipulation. Key to our hierarchical framework is that we employ structured semantic layout as our intermediate representation for manipulation. Initialized with coarse-level bounding boxes, our structure generator first creates pixel-wise semantic layout capturing the object shape, object-object interactions, and object-scene relations. Then our image generator fills in the pixel-level textures guided by the semantic layout. Such framework allows a user to manipulate images at object-level by adding, removing, and moving one bounding box at a time. Experimental evaluations demonstrate the advantages of the hierarchical manipulation framework over existing image generation and context hole-filing models, both qualitatively and quantitatively. Benefits of the hierarchical framework are further demonstrated in applications such as semantic object manipulation, interactive image editing, and data-driven image manipulation. |
Year | Venue | Keywords |
---|---|---|
2018 | ADVANCES IN NEURAL INFORMATION PROCESSING SYSTEMS 31 (NIPS 2018) | object manipulation,bounding box,image manipulation,image generator,direct manipulation |
DocType | Volume | ISSN |
Conference | 31 | 1049-5258 |
Citations | PageRank | References |
6 | 0.40 | 15 |
Authors | ||
4 |
Name | Order | Citations | PageRank |
---|---|---|---|
Seunghoon Hong | 1 | 899 | 30.34 |
Xinchen Yan | 2 | 415 | 16.71 |
Thomas S. Huang | 3 | 27815 | 2618.42 |
Honglak Lee | 4 | 6247 | 398.39 |