Paper Info
Title
One-Shot Imitation Drone Filming of Human Motion Videos
Abstract
Imitation learning has recently been applied to mimic the operation of a cameraman in existing autonomous camera systems. To imitate a certain demonstration video, existing methods require users to collect a significant number of training videos with a similar filming style. Because the trained model is style-specific, it is challenging to generalize the model to imitate other videos with a different filming style. To address this problem, we propose a framework that we term “one-shot imitation filming”, which can imitate a filming style by “seeing” only a single demonstration video of the target style without style-specific model training. This is achieved by two key enabling techniques: 1) filming style feature extraction, which encodes sequential cinematic characteristics of a variable-length video clip into a fixed-length feature vector; and 2) camera motion prediction, which dynamically plans the camera trajectory to reproduce the filming style of the demo video. We implemented the approach with a deep neural network and deployed it on a 6 degrees of freedom (DOF) drone system by first predicting the future camera motions, and then converting them into the drone's control commands via an odometer. Our experimental results on comprehensive datasets and showcases exhibit that the proposed approach achieves significant improvements over conventional baselines, and our approach can mimic the footage of an unseen style with high fidelity.
Year
DOI
Venue
2022
10.1109/TPAMI.2021.3067359
IEEE Transactions on Pattern Analysis and Machine Intelligence
Keywords
DocType
Volume
Algorithms,Humans,Imitative Behavior,Motion,Unmanned Aerial Devices,Video Recording
Journal
44
Issue
ISSN
Citations 
9
0162-8828
0
PageRank 
References 
Authors
0.34
9
5
Name
Order
Citations
PageRank
Chong Huang1182.51
Yuanjie Dang281.88
Peng Chen3147.57
Xin Yang422825.10
Kwang-Ting Cheng55755513.90