Paper Info
Title
Interaction-Aware Spatio-Temporal Pyramid Attention Networks For Action Classification
Abstract
Local features at neighboring spatial positions in feature maps have high correlation since their receptive fields are often overlapped. Self-attention usually uses the weighted sum (or other functions) with internal elements of each local feature to obtain its weight score, which ignores interactions among local features. To address this, we propose an effective interaction-aware self-attention model inspired by PCA to learn attention maps. Furthermore, since different layers in a deep network capture feature maps of different scales, we use these feature maps to construct a spatial pyramid and then utilize multi-scale information to obtain more accurate attention scores, which are used to weight the local features in all spatial positions of feature maps to calculate attention maps. Moreover, our spatial pyramid attention is unrestricted to the number of its input feature maps so it is easily extended to a spatiotemporal version. Finally, our model is embedded in general CNNs to form end-to-end attention networks for action classification. Experimental results show that our method achieves the state-of-the-art results on the UCF101, HMDB51 and untrimmed Charades.
Year
DOI
Venue
2018
10.1007/978-3-030-01270-0_23
COMPUTER VISION - ECCV 2018, PT XVI
DocType
Volume
ISSN
Conference
11220
0302-9743
Citations 
PageRank 
References 
7
0.62
37
Authors
6
Name
Order
Citations
PageRank
Yang Du1146.47
Chunfeng Yuan241830.84
Bing Li321760.28
Lili Zhao4287.86
Li Yangxi5345.75
Weiming Hu65300261.38