Cross-Attention in Coupled Unmixing Nets for Unsupervised Hyperspectral Super-Resolution - Archive ouverte HAL
Communication Dans Un Congrès Année : 2020

Cross-Attention in Coupled Unmixing Nets for Unsupervised Hyperspectral Super-Resolution

Résumé

The recent advancement of deep learning techniques has made great progress on hyperspectral image super-resolution (HSI-SR). Yet the development of unsupervised deep networks remains challenging for this task. To this end, we propose a novel coupled unmixing network with a cross-attention mechanism, CUCaNet for short, to enhance the spatial resolution of HSI by means of higher-spatial-resolution multispectral image (MSI). Inspired by coupled spectral unmixing, a two-stream convolutional autoencoder framework is taken as backbone to jointly decompose MS and HS data into a spectrally meaningful basis and corresponding coefficients. CUCaNet is capable of adaptively learning spectral and spatial response functions from HS-MS correspondences by enforcing reasonable consistency assumptions on the networks. Moreover, a cross-attention module is devised to yield more effective spatial-spectral information transfer in networks. Extensive experiments are conducted on three widely-used HS-MS datasets in comparison with state-of-the-art HSI-SR models, demonstrating the superiority of the CUCaNet in the HSI-SR application. Furthermore, the codes and datasets are made available at: https://github.com/danfenghong/ECCV2020_CUCaNet.

Dates et versions

hal-03142195 , version 1 (15-02-2021)

Identifiants

Citer

Jing Yao, Danfeng Hong, Jocelyn Chanussot, Deyu Meng, Xiaoxiang Zhu, et al.. Cross-Attention in Coupled Unmixing Nets for Unsupervised Hyperspectral Super-Resolution. ECCV 2020 - 16th European Conference on Computer Vision, Aug 2020, Glasgow, United Kingdom. pp.208-224, ⟨10.1007/978-3-030-58526-6_13⟩. ⟨hal-03142195⟩
120 Consultations
0 Téléchargements

Altmetric

Partager

More