Paper Info
Title
Magnet: Multi-Region Attention-Assisted Grounding Of Natural Language Queries At Phrase Level
Abstract
Grounding free-form textual queries necessitates an understanding of these textual phrases and its relation to the visual cues to reliably reason about the described locations. Spatial attention networks are known to learn this relationship and focus its gaze on salient objects in the image. Thus, we propose to utilize spatial attention networks for image-level visual-textual fusion preserving local (word) and global (phrase) information to refine region proposals with an in-network Region Proposal Network (RPN) and detect single or multiple regions for a phrase query. We focus only on the phrase query - ground truth pair (referring expression) for a model independent of the constraints of the datasets i.e. additional attributes, context etc. For such referring expression dataset ReferIt game, our Multi-region Attention-assisted Grounding network (MAGNet) achieves over 12% improvement over the state-of-the-art. Without the context from image captions and attribute information in Flickr3Ok Entities, we still achieve competitive results compared to the state-of-the-art.
Year
DOI
Venue
2020
10.1109/ICPR48806.2021.9412473
2020 25TH INTERNATIONAL CONFERENCE ON PATTERN RECOGNITION (ICPR)
DocType
ISSN
Citations 
Conference
1051-4651
0
PageRank 
References 
Authors
0.34
0
4
Name
Order
Citations
PageRank
Amar Shrestha1164.06
Pugdeethosapol Krittaphat200.34
haowen fang3214.59
Qinru Qiu41120102.58