Name
Abstract | ||
|---|---|---|
DNN-based cross-modal retrieval is a research hotspot to retrieve across different modalities as image and text, but existing methods often face the challenge of insufficient cross-modal training data. In single-modal scenario, similar problem is usually relieved by transferring knowledge from large-scale auxiliary datasets (as ImageNet). Knowledge from such single-modal datasets is also very useful for cross-modal retrieval, which can provide rich general semantic information that can be shared across different modalities. However, it is challenging to transfer useful knowledge from single-modal (as image) source domain to cross-modal (as image/text) target domain. Knowledge in source domain cannot be directly transferred to both two different modalities in target domain, and the inherent cross-modal correlation contained in target domain provides key hints for cross-modal retrieval which should be preserved during transfer process. This paper proposes Cross-modal Hybrid Transfer Network (CHTN) with two subnetworks: Modal-sharing transfer subnetwork utilizes the modality in both source and target domains as a bridge, for transferring knowledge to both two modalities simultaneously; Layer-sharing correlation subnetwork preserves the inherent cross-modal semantic correlation to further adapt to cross-modal retrieval task. Cross-modal data can be converted to common representation by CHTN for retrieval, and comprehensive experiment on 3 datasets shows its effectiveness. |
Year | DOI | Venue |
|---|---|---|
2017 | 10.24963/ijcai.2017/263 | IJCAI |
DocType | Volume | Citations |
Conference | abs/1706.00153 | 11 |
PageRank | References | Authors |
0.52 | 12 | 3 |
Name | Order | Citations | PageRank |
|---|---|---|---|
| Xin Huang | 1 | 135 | 5.99 |
| Yuxin Peng | 2 | 1122 | 74.90 |
| Mingkuan Yuan | 3 | 71 | 3.75 |