Abstract | ||
---|---|---|
Creative sketching or doodling is an expressive activity, where imaginative and previously unseen depictions of everyday visual objects are drawn. Creative sketch image generation is a challenging vision problem, where the task is to generate diverse, yet realistic creative sketches possessing the unseen composition of the visual-world objects. Here, we propose a novel coarse-to-fine two-stage framework, DoodleFormer, that decomposes the creative sketch generation problem into the creation of coarse sketch composition followed by the incorporation of fine-details in the sketch. We introduce graph-aware transformer encoders that effectively capture global dynamic as well as local static structural relations among different body parts. To ensure diversity of the generated creative sketches, we introduce a probabilistic coarse sketch decoder that explicitly models the variations of each sketch body part to be drawn. Experiments are performed on two creative sketch datasets: Creative Birds and Creative Creatures. Our qualitative, quantitative and human-based evaluations show that DoodleFormer outperforms the state-of-the-art on both datasets, yielding realistic and diverse creative sketches. On Creative Creatures, DoodleFormer achieves an absolute gain of 25 in Frèchet inception distance (FID) over state-of-the-art. We also demonstrate the effectiveness of DoodleFormer for related applications of text to creative sketch generation, sketch completion and house layout generation. Code is available at:
https://github.com/ankanbhunia/doodleformer
. |
Year | DOI | Venue |
---|---|---|
2022 | 10.1007/978-3-031-19790-1_21 | Computer Vision – ECCV 2022 |
DocType | ISSN | Citations |
Conference | 0302-9743 | 0 |
PageRank | References | Authors |
0.34 | 0 | 7 |
Name | Order | Citations | PageRank |
---|---|---|---|
Ankan Kumar Bhunia | 1 | 1 | 1.70 |
Salman Khan | 2 | 387 | 41.05 |
Hisham Cholakkal | 3 | 48 | 8.40 |
Muhammad Anwer Rao | 4 | 129 | 11.22 |
Fahad Shahbaz Khan | 5 | 1622 | 69.24 |
Jorma Laaksonen | 6 | 1162 | 176.93 |
Michael Felsberg | 7 | 2419 | 130.29 |