Article Dans Une Revue IEEE Transactions on Neural Networks and Learning Systems Année : 2024

Spectral–Spatial Transformer for Hyperspectral Image Sharpening

Résumé

Convolutional neural networks (CNNs) have recently achieved outstanding performance for hyperspectral (HS) and multispectral (MS) image fusion. However, CNNs cannot explore the long-range dependence for HS and MS image fusion because of their local receptive fields. To overcome this limitation, a transformer is proposed to leverage the long-range dependence from the network inputs. Because of the ability of long-range modeling, the transformer overcomes the sole CNN on many tasks, whereas its use for HS and MS image fusion is still unexplored. In this article, we propose a spectral–spatial transformer (SST) to show the potentiality of transformers for HS and MS image fusion. We devise first two branches to extract spectral and spatial features in the HS and MS images by SST blocks, which can explore the spectral and spatial long-range dependence, respectively. Afterward, spectral and spatial features are fused feeding the result back to spectral and spatial branches for information interaction. Finally, the high-resolution (HR) HS image is reconstructed by dense links from all the fused features to make full use of them. The experimental analysis demonstrates the high performance of the proposed approach compared with some state-of-the-art (SOTA) methods.

Fichier non déposé

Dates et versions

hal-04910093 , version 1 (24-01-2025)

Identifiants

Citer

Lihui Chen, Gemine Vivone, Jiayi Qin, Jocelyn Chanussot, Xiaomin Yang. Spectral–Spatial Transformer for Hyperspectral Image Sharpening. IEEE Transactions on Neural Networks and Learning Systems, 2024, 35 (11), pp.16733-16747. ⟨10.1109/TNNLS.2023.3297319⟩. ⟨hal-04910093⟩
226 Consultations
0 Téléchargements

Altmetric

Partager

  • More