Paper Info
Title
Part-Based Semantic Transform for Few-Shot Semantic Segmentation
Abstract
Few-shot semantic segmentation remains an open problem for the lack of an effective method to handle the semantic misalignment between objects. In this article, we propose part-based semantic transform (PST) and target at aligning object semantics in support images with those in query images by semantic decomposition-and-match. The semantic decomposition process is implemented with prototype mixture models (PMMs), which use an expectation–maximization (EM) algorithm to decompose object semantics into multiple prototypes corresponding to object parts. The semantic match between prototypes is performed with a min-cost flow module, which encourages correct correspondence while depressing mismatches between object parts. With semantic decomposition-and-match, PST enforces the network’s tolerance to objects’ appearance and/or pose variation and facilities channelwise and spatial semantic activation of objects in query images. Extensive experiments on Pascal VOC and MS-COCO datasets show that PST significantly improves upon state-of-the-arts. In particular, on MS-COCO, it improves the performance of five-shot semantic segmentation by up to 7.79% with a moderate cost of inference speed and model size. Code for PST is released at <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://github.com/Yang-Bob/PST</uri> .
Year
DOI
Venue
2022
10.1109/TNNLS.2021.3084252
IEEE Transactions on Neural Networks and Learning Systems
Keywords
DocType
Volume
Few-shot segmentation,prototype mixture models (PMMs),semantic match,semantic transform
Journal
33
Issue
ISSN
Citations 
12
2162-237X
0
PageRank 
References 
Authors
0.34
11
6
Name
Order
Citations
PageRank
Boyu Yang100.68
Fang Wan2213.44
Chang Liu3571117.41
Bohao Li400.68
Xiangyang Ji553373.14
Qixiang Ye691364.51