Name
Abstract | ||
|---|---|---|
This study proposes a novel
<italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">unified</i>
and
<italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">unsupervised</i>
end-to-end image
<italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">fusion</i>
network, termed as
<italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">U2Fusion</i>
, which is capable of solving different fusion problems, including multi-modal, multi-exposure, and multi-focus cases. Using feature extraction and information measurement, U2Fusion automatically estimates the importance of corresponding source images and comes up with adaptive information preservation degrees. Hence, different fusion tasks are unified in the same framework. Based on the adaptive degrees, a network is trained to preserve the adaptive similarity between the fusion result and source images. Therefore, the stumbling blocks in applying deep learning for image fusion, e.g., the requirement of ground-truth and specifically designed metrics, are greatly mitigated. By avoiding the loss of previous fusion capabilities when training a single model for different tasks sequentially, we obtain a unified model that is applicable to multiple fusion tasks. Moreover, a new aligned infrared and visible image dataset,
<italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">RoadScene</i>
(available at
<uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://github.com/hanna-xu/RoadScene</uri>
), is released to provide a new option for benchmark evaluation. Qualitative and quantitative experimental results on three typical image fusion tasks validate the effectiveness and universality of U2Fusion. Our code is publicly available at
<uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://github.com/hanna-xu/U2Fusion</uri>
. |
Year | DOI | Venue |
|---|---|---|
2022 | 10.1109/TPAMI.2020.3012548 | IEEE Transactions on Pattern Analysis and Machine Intelligence |
Keywords | DocType | Volume |
Image fusion,unified model,unsupervised learning,continual learning | Journal | 44 |
Issue | ISSN | Citations |
1 | 0162-8828 | 20 |
PageRank | References | Authors |
0.63 | 36 | 5 |
Name | Order | Citations | PageRank |
|---|---|---|---|
| Han Xu | 1 | 49 | 2.94 |
| Jiayi Ma | 2 | 1302 | 65.86 |
| Junjun Jiang | 3 | 1138 | 74.49 |
| Xiaojie Guo | 4 | 668 | 40.94 |
| Haibin Ling | 5 | 4531 | 215.76 |