Abstract | ||
---|---|---|
What constitutes an object? This has been a long-standing question in computer vision. Towards this goal, numerous learning-free and learning-based approaches have been developed to score objectness. However, they generally do not scale well across new domains and novel objects. In this paper, we advocate that existing methods lack a top-down supervision signal governed by human-understandable semantics. For the first time in literature, we demonstrate that Multi-modal Vision Transformers (MViT) trained with aligned image-text pairs can effectively bridge this gap. Our extensive experiments across various domains and novel objects show the state-of-the-art performance of MViTs to localize generic objects in images. Based on the observation that existing MViTs do not include multi-scale feature processing and usually require longer training schedules, we develop an efficient MViT architecture using multi-scale deformable attention and late vision-language fusion. We show the significance of MViT proposals in a diverse range of applications including open-world object detection, salient and camouflage object detection, supervised and self-supervised detection tasks. Further, MViTs can adaptively generate proposals given a specific language query and thus offer enhanced interactability. Code: https://git.io/J1HPY. |
Year | DOI | Venue |
---|---|---|
2022 | 10.1007/978-3-031-20080-9_30 | European Conference on Computer Vision |
Keywords | DocType | Citations |
Object detection,Class-agnostic,Vision transformers | Conference | 0 |
PageRank | References | Authors |
0.34 | 0 | 6 |
Name | Order | Citations | PageRank |
---|---|---|---|
Muhammad Maaz | 1 | 0 | 0.34 |
Hanoona Abdul Rasheed | 2 | 0 | 0.34 |
Salman Khan | 3 | 387 | 41.05 |
Fahad Shahbaz Khan | 4 | 1622 | 69.24 |
Muhammad Anwer Rao | 5 | 129 | 11.22 |
Yang Ming-Hsuan | 6 | 15303 | 620.69 |