Paper Info
Title
An Automated End-To-End Pipeline for Fine-Grained Video Annotation using Deep Neural Networks.
Abstract
The searchability of video content is often limited to the descriptions authors and/or annotators care to provide. The level of description can range from absolutely nothing to fine-grained annotations at the level of frames. Based on these annotations, certain parts of the video content are more searchable than others. Within the context of the STEAMER project, we developed an innovative end-to-end system that attempts to tackle the problem of unsupervised retrieval of news video content, leveraging multiple information streams and deep neural networks. In particular, we extracted keyphrases and named entities from transcripts, subsequently refining these keyphrases and named entities based on their visual appearance in the news video content. Moreover, to allow for fine-grained frame-level annotations, we temporally located high-confidence keyphrases in the news video content. To that end, we had to tackle challenges such as the automatic construction of training sets and the automatic assessment of keyphrase imageability. In this paper, we discuss the main components of our end-to-end system, capable of transforming textual and visual information into fine-grained video annotations.
Year
DOI
Venue
2016
10.1145/2911996.2912028
ICMR
Keywords
Field
DocType
deep neural networks, fine-grained video annotation, video retrieval
Information retrieval,Video retrieval,Computer science,End-to-end principle,Video annotation,Deep neural networks,Visual appearance
Conference
Citations 
PageRank 
References 
0
0.34
5
Authors
6
Name
Order
Citations
PageRank
Baptist Vandersmissen1385.79
Lucas Sterckx2214.83
Thomas Demeester323030.29
Jalalvand, Azarakhsh4697.71
Wesley De Neve552554.41
Rik Van de Walle62040238.28