Spatial-enhanced Multi-level Wavelet Patching in Vision Transformers - Archive ouverte HAL Accéder directement au contenu
Article Dans Une Revue IEEE Signal Processing Letters Année : 2024

Spatial-enhanced Multi-level Wavelet Patching in Vision Transformers

Résumé

By seamlessly integrating wavelet transforms into the image patching stage of ViT, we leverage the power of multi-level wavelet transforms to decompose images into a diverse array of frequency-domain features. These features, integrated with spatial characteristics at equivalent scales, enrich image details, enhancing ViT's proficiency in delineating intricate textures and distinct edges. Consequently, we registered a notable 2.7% accuracy enhancement on the ImageNet100 dataset in ViT. Our wavelet patching module, designed for versatility, seamlessly fits into various ViT derivatives without necessitating architecture modifications. This advancement has uplifted the performance of several leading vision transformers by 0.46-4.3%, preserving parameter efficiency without notable FLOPs increment.
Fichier principal
Vignette du fichier
Wu et al_2024_Spatial-enhanced Multi-level Wavelet Patching in Vision Transformers.pdf (2.08 Mo) Télécharger le fichier
supp1-3350811.pdf (75.55 Ko) Télécharger le fichier
Origine : Fichiers produits par l'(les) auteur(s)
Licence : CC BY NC - Paternité - Pas d'utilisation commerciale
Licence : CC BY NC - Paternité - Pas d'utilisation commerciale

Dates et versions

hal-04429335 , version 1 (16-05-2024)

Licence

Paternité - Pas d'utilisation commerciale

Identifiants

Citer

Fuzhi Wu, Jiasong Wu, Huazhong Shu, Guy Carrault, Lotfi Senhadji. Spatial-enhanced Multi-level Wavelet Patching in Vision Transformers. IEEE Signal Processing Letters, 2024, IEEE Signal Processing Letters, 31, pp.446-450. ⟨10.1109/lsp.2024.3350811⟩. ⟨hal-04429335⟩
13 Consultations
0 Téléchargements

Altmetric

Partager

Gmail Facebook X LinkedIn More