A Two-Stream Multiscale Deep Learning Architecture for Pan-Sharpening - Archive ouverte HAL Accéder directement au contenu
Article Dans Une Revue IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing Année : 2020

A Two-Stream Multiscale Deep Learning Architecture for Pan-Sharpening

Résumé

Pan-sharpening, which fuses the high-resolution panchromatic (PAN) image and the low-resolution multispectral image (MSI), is a hot topic in remote sensing. Recently, deep learning technology has been successfully applied in pan-sharpening. However, the existing methods ignore that the MSI and PAN image are at different resolutions and use the same networks to extract features of the two images. To address this problem, we propose a two-stream deep learning architecture, called coupled multiscale convolutional neural network, for pan-sharpening. The proposed network has three components, feature extraction subnetworks, fusion layer, and super-resolution subnetwork. In the feature extraction subnetworks, two subnetworks are used to extract the features of the MSI and PAN image separately. Different sizes of convolutional kernels are used in the first layers due to the different spatial resolutions. Thus, the source images are mapped to the similar scale. Then a multiscale asymmetric convolution factorization is used to extract features at different scales. In the fusion layer, the two feature extraction subnetworks are coupled. Features at the same scale are first summed, and then the features of all scales are concatenated as one feature map. At last, a super-resolution subnetwork is used to generate the high-resolution MSI. Experimental results on both synthetic and real data sets demonstrate that the proposed method outperforms the other state-of-the-art pan-sharpening methods.

Dates et versions

hal-03142167 , version 1 (15-02-2021)

Identifiants

Citer

Jie Wei, Yang Xu, Wanting Cai, Zebin Wu, Jocelyn Chanussot, et al.. A Two-Stream Multiscale Deep Learning Architecture for Pan-Sharpening. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2020, 13, pp.5455-5465. ⟨10.1109/JSTARS.2020.3021074⟩. ⟨hal-03142167⟩
55 Consultations
0 Téléchargements

Altmetric

Partager

Gmail Facebook X LinkedIn More