Paper Info
Title
Pan-Sharpening With Color-Aware Perceptual Loss And Guided Re-Colorization
Abstract
In remote sensing, "pan-sharpening" is the task of enhancing the spatial resolution of a multi-spectral (MS) image by exploiting the high-frequency information in a panchromatic (PAN) reference image. We present a novel color-aware perceptual (CAP) loss for learning the task of pan-sharpening. Our CAP loss is designed to focus on the deep features of a pre-trained VGG network that are more sensitive to spatial details and ignore color information to allow the network to extract the structural information from the PAN image while keeping the color from the lower resolution MS image. Additionally, we propose "guided re-colorization", which generates a pan-sharpened image with real colors from the MS input by "picking" the closest MS pixel color for each pan-sharpened pixel, as a human operator would do in manual colorization. Such a re-colorized (RC) image is completely aligned with the pan-sharpened (PS) network output and can be used as a self-supervision signal during training, or to enhance the colors in the PS image during test. We present several experiments where our network trained with our CAP loss generates naturally looking pan-sharpened images with fewer artifacts and outperforms the state-of-the-arts on the WorldView3 dataset in terms of ERGAS, SCC, and QNR metrics.
Year
DOI
Venue
2020
10.1109/ICIP40778.2020.9190785
2020 IEEE INTERNATIONAL CONFERENCE ON IMAGE PROCESSING (ICIP)
Keywords
DocType
ISSN
Pan-sharpening, pan-colorization, deep convolutional neural network (DCNN), perceptual loss, satellite imagery
Conference
1522-4880
Citations 
PageRank 
References 
0
0.34
0
Authors
3
Name
Order
Citations
PageRank
Juan Luis Gonzalez112.03
Soomin Seo261.79
Munchurl Kim385868.28