Paper Info
Title
Various Syncretic Co-Attention Network For Multimodal Sentiment Analysis
Abstract
The multimedia contents shared on social network reveal public sentimental attitudes toward specific events. Therefore, it is necessary to conduct sentiment analysis automatically on abundant multimedia data posted by the public for real-world applications. However, approaches to single-modal sentiment analysis neglect the internal connections between textual and visual contents, and current multimodal methods fail to exploit the multilevel semantic relations of heterogeneous features. In this article, the various syncretic co-attention network is proposed to excavate the intricate multilevel corresponding relations between multimodal data, and combine the unique information of each modality for integrated complementary sentiment classification. Specifically, a multilevel co-attention module is constructed to explore localized correspondences between each image region and each text word, and holistic correspondences between global visual information and context-based textual semantics. Then, all the single-modal features can be fused from different levels, respectively. Except for fused multimodal features, our proposed VSCN also considers unique information of each modality simultaneously and integrates them into an end-to-end framework for sentiment analysis. The superior results of experiments on three constructed real-world datasets and a benchmark dataset of Visual Sentiment Ontology (VSO) prove the effectiveness of our proposed VSCN. Especially qualitative analyses are given for deep explaining of our method.
Year
DOI
Venue
2020
10.1002/cpe.5954
CONCURRENCY AND COMPUTATION-PRACTICE & EXPERIENCE
Keywords
DocType
Volume
co-attention network, multilevel, multimodal, sentiment analysis
Journal
32
Issue
ISSN
Citations 
24
1532-0626
0
PageRank 
References 
Authors
0.34
0
5
Name
Order
Citations
PageRank
Meng Cao100.34
Yonghua Zhu200.68
Wenjing Gao301.01
Mengyao Li401.35
Shaoxiu Wang500.34