Name
Abstract | ||
|---|---|---|
We present a novel approach to enhance the challenging task of Visual Question Answering (VQA) by incorporating and enriching semantic knowledge in a VQA model. We first apply Multiple Instance Learning (MIL) to extract a richer visual representation addressing concepts beyond objects such as actions and colors. Motivated by the observation that semantically related answers often appear together in prediction, we further develop a new semantically-guided loss function for model learning which has the potential to drive weakly-scored but correct answers to the top while suppressing wrong answers. We show that these two ideas contribute to performance improvement in a complementary way. We demonstrate competitive results comparable to the state of the art on two VQA benchmark datasets. |
Year | DOI | Venue |
|---|---|---|
2018 | 10.1109/WACV.2018.00205 | 2018 IEEE WINTER CONFERENCE ON APPLICATIONS OF COMPUTER VISION (WACV 2018) |
Field | DocType | ISSN |
Semantic memory,Question answering,Task analysis,Pattern recognition,Visualization,Computer science,Feature extraction,Artificial intelligence,Natural language processing,Semantics,Performance improvement,Model learning | Conference | 2472-6737 |
Citations | PageRank | References |
1 | 0.35 | 0 |
Authors | ||
4 | ||
Name | Order | Citations | PageRank |
|---|---|---|---|
| Handong Zhao | 1 | 46 | 3.66 |
| Quanfu Fan | 2 | 504 | 32.69 |
| Dan Gutfreund | 3 | 1 | 0.35 |
| Yun Fu | 4 | 4267 | 208.09 |