Name
Abstract | ||
|---|---|---|
Temporal grounding aims to localize a video moment which is semantically aligned with a given natural language query. Existing methods typically apply a detection or regression pipeline on the fused representation with the research focus on designing complicated prediction heads or fusion strategies. Instead, from a perspective on temporal grounding as a metric-learning problem, we present a Mutual Matching Network (MMN), to directly model the similarity between language queries and video moments in a joint embedding space. This new metric-learning framework enables fully exploiting negative samples from two new aspects: constructing negative cross-modal pairs in a mutual matching scheme and mining negative pairs across different videos. These new negative samples could enhance the joint representation learning of two modalities via cross-modal mutual matching to maximize their mutual information. Experiments show that our MMN achieves highly competitive performance compared with the state-of-the-art methods on four video grounding benchmarks. Based on MMN, we present a winner solution for the HC-STVG challenge of the 3rd PIC workshop. This suggests that metric learning is still a promising method for temporal grounding via capturing the essential cross-modal correlation in a joint embedding space. Code is available at https://github.com/MCG-NJU/MMN. |
Year | Venue | Keywords |
|---|---|---|
2022 | AAAI Conference on Artificial Intelligence | Computer Vision (CV),Speech & Natural Language Processing (SNLP) |
DocType | Citations | PageRank |
Conference | 0 | 0.34 |
References | Authors | |
0 | 5 | |
Name | Order | Citations | PageRank |
|---|---|---|---|
| Zhenzhi Wang | 1 | 0 | 0.34 |
| LiMin Wang | 2 | 816 | 48.41 |
| Tao Wu | 3 | 0 | 0.34 |
| Tianhao Li | 4 | 0 | 0.34 |
| Gang-Shan Wu | 5 | 27 | 6.75 |