Paper Info
Title
Semantically Similarity-Wise Dual-Branch Network for Scene Graph Generation
Abstract
Scene graph generation aims to detect visual entities and relationships between them from an image. The object-level visual information is of vital importance for predicting accurate relationships. However, most existing methods essentially encode visual information with coarse supervised information, since they regard different relationships as mutually exclusive semantics with equal-distance labels by taking cross-entropy function as the main training loss. Intuitively, different relationship semantics naturally have their own similarity and dissimilarity with different level distances, <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">i.e.</i> , the topological information of relationship semantics. It can serve as an inspiring hint to aid learning to grasp the key related visual information. Accordingly, we propose a Semantically Similarity-wise Dual-branch Network (SSDN) which introduces topological information of relationship semantics as extra supervision to aid learning extracting and encoding relationship-related visual information. To avoid possible chaotic feature learning and enable the introduced knowledge to be better absorbed during inference, we design a dual-branch framework consisting of an auxiliary branch and an inference branch. The topological information extracted from the groundtruth is introduced at the front end of the auxiliary branch which then generates a soft embedding to be propagated to the inference branch in a knowledge distillation manner. Extensive experiments show that our model averagely outperforms state-of-the-art approaches on benchmark Visual Genome and VRD significantly, which demonstrates its effectiveness and superiority.
Year
DOI
Venue
2022
10.1109/TCSVT.2021.3130197
IEEE Transactions on Circuits and Systems for Video Technology
Keywords
DocType
Volume
Scene graph generation,visual relationship detection,message propagating
Journal
32
Issue
ISSN
Citations 
7
1051-8215
0
PageRank 
References 
Authors
0.34
16
5
Name
Order
Citations
PageRank
Bowen Zhao100.34
Zhendong Mao230725.18
Shancheng Fang300.34
Wenyu Zang400.34
Yongdong Zhang52544166.91