Paper Info
Title
TimeReplayer: Unlocking the Potential of Event Cameras for Video Interpolation
Abstract
Recording fast motion in a high FPS (frame-per-second) requires expensive high-speed cameras. As an alternative, interpolating low-FPS videos from commodity cameras has attracted significant attention. If only low-FPS videos are available, motion assumptions (linear or quadratic) are necessary to infer intermediate frames, which fail to model complex motions. Event camera, a new camera with pixels producing events of brightness change at the temporal resolution of μs (10– 6 second), is a game-changing device to enable video interpolation at the presence of arbitrarily complex motion. Since event camera is a novel sensor, its potential has not been fulfilled due to the lack of processing algorithms. The pioneering work Time Lens introduced event cameras to video interpolation by designing optical devices to collect a large amount of paired training data of high-speed frames and events, which is too costly to scale. To fully unlock the potential of event cameras, this paper proposes a novel TimeReplayer algorithm to interpolate videos captured by commodity cameras with events. It is trained in an unsupervised cycleconsistent style, canceling the necessity of high-speed training data and bringing the additional ability of video extrapolation. Its state-of-the-art results and demo videos in supplementary reveal the promising future of event-based vision.
Year
DOI
Venue
2022
10.1109/CVPR52688.2022.01728
IEEE Conference on Computer Vision and Pattern Recognition
Keywords
DocType
Volume
Computational photography, Low-level vision
Conference
2022
Issue
Citations 
PageRank 
1
0
0.34
References 
Authors
0
9
Name
Order
Citations
PageRank
Weihua He121.06
Kaichao You200.34
Zhendong Qiao300.34
Xu Jia433320.97
Ziyang Zhang500.68
Wenhui Wang600.34
Huchuan Lu74827186.26
Yaoyuan Wang801.69
Jianxing Liao901.01