Name
Abstract | ||
|---|---|---|
Artificial creativity has attracted increasing research attention in the field of multimedia and artificial intelligence. Despite the promising work on poetry/painting/music generation, creating modern Chinese poetry from images, which can significantly enrich the functionality of photo-sharing platforms, has rarely been explored. Moreover, existing generation models cannot tackle three challenges in this task: (1) Maintaining semantic consistency between images and poems; (2) preventing topic drift in the generation; (3) avoidance of certain words appearing frequently. These three points are even common challenges in other sequence generation tasks. In this article, we propose a Constrained Topic-aware Model (CTAM) to create modern Chinese poetries from images regarding the challenges above. Without image-poetry paired dataset, we construct a visual semantic vector to embed visual contents via image captions. For the topic-drift problem, we propose a topic-aware poetry generation model. Additionally, we design an Anti-frequency Decoding (AFD) scheme to constrain high-frequency characters in the generation. Experimental results show that our model achieves promising performance and is effective in poetry’s readability and semantic consistency.
|
Year | DOI | Venue |
|---|---|---|
2020 | 10.1145/3381858 | ACM Transactions on Multimedia Computing, Communications, and Applications |
Keywords | DocType | Volume |
Image captioning,poetry generation,semantic consistency | Journal | 16 |
Issue | ISSN | Citations |
2 | 1551-6857 | 1 |
PageRank | References | Authors |
0.36 | 0 | 4 |
Name | Order | Citations | PageRank |
|---|---|---|---|
| Lingxiang Wu | 1 | 8 | 0.82 |
| Min Xu | 2 | 268 | 28.42 |
| Shengsheng Qian | 3 | 130 | 19.10 |
| Jianwei Cui | 4 | 5 | 5.23 |