Pay attention to your loss: understanding misconceptions about 1-Lipschitz neural networks - Archive ouverte HAL Accéder directement au contenu
Communication Dans Un Congrès Année : 2022

Pay attention to your loss: understanding misconceptions about 1-Lipschitz neural networks

Résumé

Lipschitz constrained networks have gathered considerable attention in the deep learning community, with usages ranging from Wasserstein distance estimation to the training of certifiably robust classifiers. However they remain commonly considered as less accurate, and their properties in learning are still not fully understood. In this paper we clarify the matter: when it comes to classification 1-Lipschitz neural networks enjoy several advantages over their unconstrained counterpart. First, we show that these networks are as accurate as classical ones, and can fit arbitrarily difficult boundaries. Then, relying on a robustness metric that reflects operational needs we characterize the most robust classifier: the WGAN discriminator. Next, we show that 1-Lipschitz neural networks generalize well under milder assumptions. Finally, we show that hyper-parameters of the loss are crucial for controlling the accuracy-robustness trade-off. We conclude that they exhibit appealing properties to pave the way toward provably accurate, and provably robust neural networks.

Dates et versions

hal-03872080 , version 1 (25-11-2022)

Identifiants

Citer

Louis Béthune, Thibaut Boissin, Mathieu Serrurier, Franck Mamalet, Corentin Friedrich, et al.. Pay attention to your loss: understanding misconceptions about 1-Lipschitz neural networks. Advances in Neural Information Processing Systems, Nov 2022, New Orleans, United States. ⟨10.48550/arXiv.2104.05097⟩. ⟨hal-03872080⟩
74 Consultations
0 Téléchargements

Altmetric

Partager

Gmail Facebook X LinkedIn More