Paper Info
Title
Spatio-Temporal Representation Matching-Based Open-Set Action Recognition By Joint Learning Of Motion And Appearance
Abstract
In this paper, we propose the spatio-temporal representation matching (STRM) for video-based action recognition under the open-set condition. Open-set action recognition is a more challenging problem than closed-set action recognition since samples of the untrained action class need to be recognized and most of the conventional frameworks are likely to give a false prediction. To handle the untrained action classes, we propose STRM, which involves jointly learning both motion and appearance. STRM extracts spatio-temporal representations from video clips through a joint learning pipeline with both motion and appearance information. Then, STRM computes the similarities between the ST-representations to find the one with highest similarity. We set the experimental protocol for open-set action recognition and carried out experiments on UCF101 and HMDB51 to evaluate STRM. We first investigated the effects of different hyper-parameter settings on STRM, and then compared its performance with existing state-of-the-art methods. The experimental results showed that the proposed method not only outperformed existing methods under the open-set condition, but also provided comparable performance to the state-of-the-art methods under the closed-set condition.
Year
DOI
Venue
2019
10.1109/ACCESS.2019.2953455
IEEE ACCESS
Keywords
DocType
Volume
Action recognition, open-set recognition, spatio-temporal representation, joint learning of motion and appearance
Journal
7
ISSN
Citations 
PageRank 
2169-3536
0
0.34
References 
Authors
0
3
Name
Order
Citations
PageRank
Yongsang Yoon100.68
Jongmin Yu294.54
Moongu Jeon345672.81