Paper Info
Title
Exploring complementary information of self-supervised pretext tasks for unsupervised video pre-training
Abstract
This study addresses the problem of the unsupervised pre-training of video representation learning. The authors' focus is on two common approaches: knowledge distillation and self-supervised learning. The insight provided is that knowledge distillation and the rapidly advancing self-supervised technique can be mutually beneficial. Combining these two approaches, a unified framework of self-supervised learning and image-based distillation (SSID) for unsupervised video pre-training is proposed. The effectiveness of SSID in comparison to both image-based distillation methods and the existing self-supervised pre-training baseline is demonstrated. In particular, the authors' model leverages three signals from the unlabelled data. First, the authors distil from the classifier of a 2D pre-trained model as a soft label. To regularize the training process, the authors then build a novel positive pair of contrastive learning on the representation of the 2D/3D model. Finally, a self-supervised pretext task is introduced to enhance the authors' model to become aware of the temporal evolution. The authors' experiment results showed that the learnt features achieved the best performance when transferred to action recognition tasks on UCF101 and HMDB51, reaching increases of 2.4% and 1.9% compared to the existing unsupervised pre-training model, respectively.
Year
DOI
Venue
2022
10.1049/cvi2.12084
IET COMPUTER VISION
Keywords
DocType
Volume
computer vision, convolutional neural nets, feature extraction, unsupervised learning, video signal processing, image sequences
Journal
16
Issue
ISSN
Citations 
3
1751-9632
0
PageRank 
References 
Authors
0.34
0
4
Name
Order
Citations
PageRank
Wei Zhou112254.40
Yi Hou202.37
Kewei Ouyang301.69
Shilin Zhou47213.94