Exploring Machine Learning Privacy/Utility Trade-Off from a Hyperparameters Lens - Archive ouverte HAL Accéder directement au contenu
Communication Dans Un Congrès Année : 2023

Exploring Machine Learning Privacy/Utility Trade-Off from a Hyperparameters Lens

Résumé

Machine Learning (ML) architectures have been applied to several applications that involve sensitive data, where a guarantee of users' data privacy is required. Differentially Private Stochastic Gradient Descent (DPSGD) is the state-of-the-art method to train privacy-preserving models. However, DPSGD comes at a considerable accuracy loss leading to sub-optimal privacy/utility trade-offs. Towards investigating new ground for better privacy-utility trade-off, this work questions; (i) if models' hyperparameters have any inherent impact on ML models' privacy-preserving properties, and (ii) if models' hyperparameters have any impact on the privacy/utility trade-off of differentially private models. We propose a comprehensive design space exploration of different hyperparameters such as the choice of activation functions, the learning rate and the use of batch normalization. Interestingly, we found that utility can be improved by using Bounded RELU as activation functions with the same privacy-preserving characteristics. With a drop-in replacement of the activation function, we achieve new state-of-the-art accuracy on MNIST (96.02%), FashionMnist (84.76%), and CIFAR-10 (44.42%) without any modification of the learning procedure fundamentals of DPSGD.
Fichier principal
Vignette du fichier
Ayoub Arous_2023_2303.01819.pdf (2.1 Mo) Télécharger le fichier
Origine : Fichiers produits par l'(les) auteur(s)

Dates et versions

hal-04202435 , version 1 (11-09-2023)

Identifiants

Citer

Ayoub Arous, Amira Guesmi, Muhammad Abdullah Hanif, Ihsen Alouani, Muhammad Shafique. Exploring Machine Learning Privacy/Utility Trade-Off from a Hyperparameters Lens. International Joint Conference on Neural Networks (IJCNN 2023), Jun 2023, Gold Coast, Australia. ⟨10.1109/IJCNN54540.2023.10191743⟩. ⟨hal-04202435⟩
13 Consultations
19 Téléchargements

Altmetric

Partager

Gmail Facebook X LinkedIn More