Spatial-enhanced Multi-level Wavelet Patching in Vision Transformers
Résumé
By seamlessly integrating wavelet transforms into the image patching stage of ViT, we leverage the power of multi-level wavelet transforms to decompose images into a diverse array of frequency-domain features. These features, integrated with spatial characteristics at equivalent scales, enrich image details, enhancing ViT's proficiency in delineating intricate textures and distinct edges. Consequently, we registered a notable 2.7% accuracy enhancement on the ImageNet100 dataset in ViT. Our wavelet patching module, designed for versatility, seamlessly fits into various ViT derivatives without necessitating architecture modifications. This advancement has uplifted the performance of several leading vision transformers by 0.46-4.3%, preserving parameter efficiency without notable FLOPs increment.
Fichier principal
Wu et al_2024_Spatial-enhanced Multi-level Wavelet Patching in Vision Transformers.pdf (2.08 Mo)
Télécharger le fichier
supp1-3350811.pdf (75.55 Ko)
Télécharger le fichier
Origine : Fichiers produits par l'(les) auteur(s)
Licence : CC BY NC - Paternité - Pas d'utilisation commerciale
Licence : CC BY NC - Paternité - Pas d'utilisation commerciale
Licence : CC BY NC - Paternité - Pas d'utilisation commerciale