Paper Info
Title
Evaluating Image-Inspired Poetry Generation
Abstract
Creative natural language generation, such as poetry generation, writing lyrics, and storytelling, is appealing but difficult to evaluate. We take the application of image-inspired poetry generation as a showcase and investigate two problems in evaluation: (1) how to evaluate the generated text when there are no ground truths, and (2) how to evaluate nondeterministic systems that output different texts given the same input image. Regarding the first problem, we first design a judgment tool to collect ratings of a few poems for comparison with the inspiring image shown to assessors. We then propose a novelty measurement that quantifies how different a generated text is compared to a known corpus. Regarding the second problem, we experiment with different strategies to approximate evaluating multiple trials of output poems. We also use a measure for quantifying the diversity of different texts generated in response to the same input image, and discuss their merits.
Year
DOI
Venue
2019
10.1007/978-3-030-32233-5_42
Lecture Notes in Artificial Intelligence
Keywords
DocType
Volume
Evaluation,Poetry generation,Natural language generation,AI-based creation,Image
Conference
11838
ISSN
Citations 
PageRank 
0302-9743
1
0.36
References 
Authors
0
6
Name
Order
Citations
PageRank
Chao-Chung Wu110.36
Ruihua Song2113859.33
Tetsuya Sakai31460139.97
Wen-Feng Cheng482.53
Xing Xie59105527.49
Shou-De Lin670684.81