Normalization-Equivariant Neural Networks with Application to Image Denoising - Archive ouverte HAL
Communication Dans Un Congrès Année : 2023

Normalization-Equivariant Neural Networks with Application to Image Denoising

Résumé

In many information processing systems, it may be desirable to ensure that any change of the input, whether by shifting or scaling, results in a corresponding change in the system response. While deep neural networks are gradually replacing all traditional automatic processing methods, they surprisingly do not guarantee such normalization-equivariance (scale + shift) property, which can be detrimental in many applications. To address this issue, we propose a methodology for adapting existing neural networks so that normalization-equivariance holds by design. Our main claim is that not only ordinary convolutional layers, but also all activation functions, including the ReLU (rectified linear unit), which are applied element-wise to the pre-activated neurons, should be completely removed from neural networks and replaced by better conditioned alternatives. To this end, we introduce affine-constrained convolutions and channel-wise sort pooling layers as surrogates and show that these two architectural modifications do preserve normalization-equivariance without loss of performance. Experimental results in image denoising show that normalization-equivariant neural networks, in addition to their better conditioning, also provide much better generalization across noise levels.
Fichier principal
Vignette du fichier
NENN_arXiv-9.pdf (1.44 Mo) Télécharger le fichier
Origine Fichiers produits par l'(les) auteur(s)

Dates et versions

hal-04123516 , version 1 (09-06-2023)

Licence

Identifiants

Citer

Sébastien Herbreteau, Emmanuel Moebel, Charles Kervrann. Normalization-Equivariant Neural Networks with Application to Image Denoising. Conference on Neural Information Processing Systems (NeurIPS), Dec 2023, New-Orleans, United States. ⟨10.48550/arXiv.2306.05037⟩. ⟨hal-04123516⟩
54 Consultations
35 Téléchargements

Altmetric

Partager

More