Title | ||
---|---|---|
Mask-guided Spectral-wise Transformer for Efficient Hyperspectral Image Reconstruction |
Abstract | ||
---|---|---|
Hyperspectral image (HSI) reconstruction aims to recover the 3D spatial-spectral signal from a 2D measurement in the coded aperture snapshot spectral imaging (CASSI) system. The HSI representations are highly similar and correlated across the spectral dimension. Modeling the inter-spectra interactions is beneficial for HSI reconstruction. However, existing CNN-based methods show limitations in capturing spectral-wise similarity and long-range dependencies. Besides, the HSI information is modulated by a coded aperture (physical mask) in CASSI. Nonetheless, current algorithms have not fully explored the guidance effect of the mask for HSI restoration. In this paper, we propose a novel framework, Mask-guided Spectral-wise Transformer (MST), for HSI reconstruction. Specifically, we present a Spectral-wise Multi-head Self-Attention (S-MSA) that treats each spectral feature as a token and calculates self-attention along the spectral dimension. In addition, we customize a Mask-guided Mechanism (MM) that directs S- MSA to pay attention to spatial regions with high-fidelity spectral representations. Extensive experiments show that our MST significantly outperforms state-of-the-art (SOTA) methods on simulation and real HSI datasets while requiring dramatically cheaper computational and memory costs. https://github.com/caiyuanhao1998/MST/ |
Year | DOI | Venue |
---|---|---|
2022 | 10.1109/CVPR52688.2022.01698 | IEEE Conference on Computer Vision and Pattern Recognition |
Keywords | DocType | Volume |
Low-level vision, Computational photography | Conference | 2022 |
Issue | Citations | PageRank |
1 | 0 | 0.34 |
References | Authors | |
0 | 8 |
Name | Order | Citations | PageRank |
---|---|---|---|
Yuanhao Cai | 1 | 4 | 3.43 |
Jing Lin | 2 | 0 | 1.69 |
Xiaowan Hu | 3 | 4 | 3.09 |
Wang H | 4 | 71 | 29.35 |
Xin Yuan | 5 | 1089 | 92.27 |
Zhang Yulun | 6 | 206 | 22.15 |
Radu Timofte | 7 | 1880 | 118.45 |
Luc Van Gool | 8 | 27566 | 1819.51 |