A Novel Information-Theoretic Objective to Disentangle Representations for Fair Classification - Archive ouverte HAL Accéder directement au contenu
Communication Dans Un Congrès Année : 2023

A Novel Information-Theoretic Objective to Disentangle Representations for Fair Classification

Résumé

One of the pursued objectives of deep learning is to provide tools that learn abstract representations of reality from the observation of multiple contextual situations. More precisely, one wishes to extract disentangled representations which are (i) low dimensional and (ii) whose components are independent and correspond to concepts capturing the essence of the objects under consideration (Locatello et al., 2019b). One step towards this ambitious project consists in learning disentangled representations with respect to a predefined (sensitive) attribute, e.g., the gender or age of the writer. Perhaps one of the main application for such disentangled representations is fair classification. Existing methods extract the last layer of a neural network trained with a loss that is composed of a cross-entropy objective and a disentanglement regularizer. In this work, we adopt an information-theoretic view of this problem which motivates a novel family of regularizers that minimizes the mutual information between the latent representation and the sensitive attribute conditional to the target. The resulting set of losses, called CLINIC, is parameter free and thus, it is easier and faster to train. CLINIC losses are studied through extensive numerical experiments by training over 2k neural networks. We demonstrate that our methods offer a better disentanglement/accuracy trade-off than previous techniques, and generalize better than training with cross-entropy loss solely provided that the disentanglement task is not too constraining.
Fichier principal
Vignette du fichier
2310.13990v1.pdf (1.3 Mo) Télécharger le fichier
Origine Fichiers produits par l'(les) auteur(s)

Dates et versions

hal-04574970 , version 1 (14-05-2024)

Identifiants

Citer

Pierre Colombo, Nathan Noiry, Guillaume Staerman, Pablo Piantanida. A Novel Information-Theoretic Objective to Disentangle Representations for Fair Classification. IJCNLP-AACL 2023 : The 13th International Joint Conference on Natural Language Processing and the 3rd Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics, Nov 2023, Nusa Dua, Bali, Indonesia. pp.184-198, ⟨10.18653/v1/2023.findings-ijcnlp.17⟩. ⟨hal-04574970⟩
21 Consultations
4 Téléchargements

Altmetric

Partager

Gmail Mastodon Facebook X LinkedIn More