Name
Abstract | ||
|---|---|---|
Video generation is a challenging task due to the extremely high-dimensional distribution of the solution space. Good constraints in the solution domain would thus reduce the difficulty of approximating optimal solutions. In this paper, instead of directly generating high-dimensional video data, we propose using object landmarks as explicit structure constraints to address this issue. Specifically, we propose a two-stage framework for an action-conditioned video generation task. In our framework, the first stage aims to generate landmark sequences according to predefined motion types, and a recurrent model (RNN/LSTM) is adopted for this purpose. The landmark sequence can be regarded as a low-dimensional structure embedding of high-dimensional video data, and generating landmark sequences is much easier than generating videos. The second stage is inspired by a conditional generative adversarial network (CGAN), and we take the generated landmark sequence as a structure condition to learn a landmark-to-image translation network. Such a one-to-one translation framework avoids the difficulty of generating videos and instead transfers the video generation task to image generation, which is resolvable due to the maturity of current GAN-based models. The experimental results demonstrate that our model not only achieves promising results on rigid/nonrigid motion generation tasks but also can be extended to multiobject motion situations. |
Year | DOI | Venue |
|---|---|---|
2019 | 10.1109/tmm.2018.2885235 | IEEE Transactions on Multimedia |
Keywords | Field | DocType |
Task analysis,Gallium nitride,Image generation,Biological system modeling,Computational modeling,Strain,Adaptation models | Computer vision,Image generation,Generative adversarial network,Embedding,Task analysis,Pattern recognition,Computer science,Motion generation,Artificial intelligence,Landmark | Journal |
Volume | Issue | ISSN |
21 | 7 | 1520-9210 |
Citations | PageRank | References |
1 | 0.35 | 0 |
Authors | ||
5 | ||
Name | Order | Citations | PageRank |
|---|---|---|---|
| Yichao Yan | 1 | 90 | 6.70 |
| Bingbing Ni | 2 | 1421 | 82.90 |
| Wendong Zhang | 3 | 15 | 1.85 |
| Jingwei Xu | 4 | 20 | 5.05 |
| Xiaokang Yang | 5 | 3581 | 238.09 |