Name
Abstract | ||
|---|---|---|
Automatically generating the summary of a video is a challenging problem due to its subjective nature. Most of the previous works in the field consider the entire video to extract out the important frames. Unlike them, our paper presents MerryGoRoundNet, a supervised learning approach to solve this problem in an online fashion. We observe that to effectively summarize a video, one needs to take into account both the spatial and temporal relations between video frames. MerryGoRoundNet utilizes encoder-decoder style architecture and convolutional LSTM to establish spatiotemporal relationship and generates the summary on the fly, thereby being more efficient than non-autoregressive counterparts in terms of time and memory. In order to make summary more diverse and complete, we augment our network with unsupervised task of next frame prediction and a supervised task of scene start detection and propose a loss function that explicitly focuses on achieving the right balance between continuity and diversity in the produced summary. Ablation study performed affirms the architecture and learning objective of our approach. Evaluation of MerryGoRoundNet on different datasets demonstrates superior performance among online summarization approaches and competitive performance when compared with offline approaches as well. |
Year | DOI | Venue |
|---|---|---|
2019 | 10.1109/WACV.2019.00056 | 2019 IEEE Winter Conference on Applications of Computer Vision (WACV) |
Keywords | Field | DocType |
Streaming media,Task analysis,Decoding,Convolutional codes,Feature extraction,Neural networks,Cost function | Automatic summarization,Architecture,Convolutional code,Task analysis,Pattern recognition,Computer science,Feature extraction,Supervised learning,Artificial intelligence,Decoding methods,Artificial neural network,Machine learning | Conference |
ISSN | ISBN | Citations |
2472-6737 | 978-1-7281-1975-5 | 0 |
PageRank | References | Authors |
0.34 | 0 | 3 |
Name | Order | Citations | PageRank |
|---|---|---|---|
| Shamit Lal | 1 | 0 | 1.01 |
| Shivam Duggal | 2 | 0 | 0.34 |
| Indu Sreedevi | 3 | 2 | 1.73 |