Paper Info
Title
Slot-VPS: Object-centric Representation Learning for Video Panoptic Segmentation
Abstract
Video Panoptic Segmentation (VPS) aims at assigning a class label to each pixel, uniquely segmenting and identifying all object instances consistently across all frames. Classic solutions usually decompose the VPS task into several subtasks and utilize multiple surrogates (e.g. boxes and masks, centers and offsets) to represent objects. However, this divide-and-conquer strategy requires complex post-processing in both spatial and temporal domains and is vulnerable to failures from surrogate tasks. In this paper, inspired by object-centric learning which learns compact and robust object representations, we present Slot- VPS, the first end-to-end framework for this task. We encode all panoptic entities in a video, including both foreground instances and background semantics, with a unified representation called panoptic slots. The coherent spatio-temporal object's information is retrieved and encoded into the panoptic slots by the proposed Video Panoptic Retriever, enabling to localize, segment, differentiate, and associate objects in a unified manner. Finally, the output panoptic slots can be directly converted into the class, mask, and object ID of panoptic objects in the video. We conduct extensive ablation studies and demonstrate the effectiveness of our approach on two benchmark datasets, Cityscapes- VP S (val and test sets) and VIPER (val set), achieving new state-of-the-art performance of 63.7, 63.3 and 56.2 VPQ, respectively.
Year
DOI
Venue
2022
10.1109/CVPR52688.2022.00310
IEEE Conference on Computer Vision and Pattern Recognition
Keywords
DocType
Volume
Video analysis and understanding, Motion and tracking
Conference
2022
Issue
Citations 
PageRank 
1
0
0.34
References 
Authors
0
9
Name
Order
Citations
PageRank
Yi Zhou100.34
Hui Zhang240371.41
Hana Lee300.34
Shuyang Sun451.13
Pingjun Li500.34
Yangguang Zhu600.34
ByungIn Yoo7365.52
Xiaojuan Qi800.34
Jae-Joon Han97412.34