Paper Info
Title
Learning To Film From Professional Human Motion Videos
Abstract
We investigate the problem of 6 degrees of freedom (DOE) camera planning for filming professional human motion videos using a camera drone. Existing methods [4, 3, 5] either plan motions for only a pan-tilt-zoom (PTZ) camera, or adopt ad-hoc solutions without carefully considering the impact of video contents and previous camera motions on the future camera motions. As a result, they can hardly achieve satisfactory results in our drone cinematography task. In this study, we propose a learning-based framework which incorporates the video contents and previous camera motions to predict the future camera motions that enable the capture of professional videos. Specifically, the inputs of our framework are video contents which are represented using subject-related feature based on 2D skeleton and scene-related features extracted from background RGB images, and camera motions which are represented using optical flows. The correlation between the inputs and output future camera motions are learned via a sequence-to-sequence convolutional long short-term memory (Seq2Seq ConvLSTM) network from a large set of video clips. We deploy our approach to a real drone system by first predicting the future camera motions, and then converting them to the drone's control commands via an odometer. Our experimental results on extensive datasets and showcases exhibit significant improvements in our approach over conventional baselines and our approach can successfully mimic the footage of a professional cameraman.
Year
DOI
Venue
2019
10.1109/CVPR.2019.00437
2019 IEEE/CVF CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION (CVPR 2019)
Field
DocType
ISSN
Computer vision,Computer science,Human motion,Artificial intelligence
Conference
1063-6919
Citations 
PageRank 
References 
0
0.34
0
Authors
7
Name
Order
Citations
PageRank
Chong Huang183.17
Chuan-En Lin211.70
Zhenyu Yang3253.81
Yan Kong400.68
Peng Chen5147.57
Xin Yang622825.10
Kwang-Ting Cheng75755513.90