Paper Info
Title
Representations for Content Creation, Manipulation and Animation
Abstract
ABSTRACTWhat I cannot create, I do not understand" said the famous writing on Dr. Feynman's blackboard. The ability to create or to change objects requires us to understand their structure and factors of variation. For example, to draw a face an artist is required to know its composition and have a good command of drawing skills (the latter is particularly challenging for the presenter). Animation additionally requires the knowledge of rigid and non-rigid motion patterns of the object. This talk shows that generation, manipulation and animation skills of deep generative models substantially benefit from such understanding. Moreover we see, the better the models can explain the data they see during training, the higher quality content they are able to generate. Understanding and generation form a loop in which improved understanding improves generation, improving understanding even more. To show this, I detail our works in three areas: video synthesis and prediction, image animation by motion retargeting. I will further introduce a new direction in video generation which allows the user to play videos as they're generated. In each of these works, the internal representation was designed to facilitate better understanding of the task, resulting in improved generation abilities. Without a single labeled example, our models are able to understand factors of variation, object parts, their shapes, their motion patterns and perform creative manipulations previously only available to trained professionals equipped with specialized software and hardware.
Year
DOI
Venue
2021
10.1145/3476099.3482882
International Multimedia Conference
DocType
Citations 
PageRank 
Conference
0
0.34
References 
Authors
0
1
Name
Order
Citations
PageRank
Sergey Tulyakov1289.28