Name
Abstract | ||
|---|---|---|
We present a system to automatically generate natural language descriptions from images. This system consists of two parts. The first part, content planning, smooths the output of computer vision-based detection and recognition algorithms with statistics mined from large pools of visually descriptive text to determine the best content words to use to describe an image. The second step, surface realization, chooses words to construct natural language sentences based on the predicted content and general statistics from natural language. We present multiple approaches for the surface realization step and evaluate each using automatic measures of similarity to human generated reference descriptions. We also collect forced choice human evaluations between descriptions from the proposed generation system and descriptions from competing approaches. The proposed system is very effective at producing relevant sentences for images. It also generates descriptions that are notably more true to the specific image content than previous work. |
Year | DOI | Venue |
|---|---|---|
2013 | 10.1109/TPAMI.2012.162 | Pattern Analysis and Machine Intelligence, IEEE Transactions |
Keywords | Field | DocType |
computer vision,data mining,image recognition,natural language processing,text analysis,BabyTalk,best content words,computer vision-based detection algorithm,computer vision-based recognition algorithm,content planning,forced choice human evaluations,image descriptions,natural language description generation,natural language sentences,statistics mining,surface realization step,visually descriptive text,Computer vision,image description generation | Computer vision,Computer science,Two-alternative forced choice,Image content,Context awareness,Image segmentation,Natural language,Artificial intelligence,Natural language processing,Recognition algorithm | Journal |
Volume | Issue | ISSN |
35 | 12 | 0162-8828 |
Citations | PageRank | References |
144 | 4.42 | 35 |
Authors | ||
4 | ||
Name | Order | Citations | PageRank |
|---|---|---|---|
| Kulkarni, G. | 1 | 144 | 4.42 |
| Premraj, V. | 2 | 144 | 4.42 |
| Vicente Ordonez | 3 | 1418 | 69.65 |
| Dhar, S. | 4 | 144 | 4.42 |