Manipulating Voice Attributes by Adversarial Learning of Structured Disentangled Representations - Archive ouverte HAL Accéder directement au contenu
Article Dans Une Revue Entropy Année : 2023

Manipulating Voice Attributes by Adversarial Learning of Structured Disentangled Representations

Laurent Benaroya
Nicolas Obin
Axel Roebel

Résumé

Voice conversion (VC) consists of digitally altering the voice of an individual to manipulate part of its content, primarily its identity, while maintaining the rest unchanged. Research in neural VC has accomplished considerable breakthroughs with the capacity to falsify a voice identity using a small amount of data with a highly realistic rendering. This paper goes beyond voice identity manipulation and presents an original neural architecture that allows the manipulation of voice attributes (e.g., gender and age). The proposed architecture is inspired by the fader network, transferring the same ideas to voice manipulation. The information conveyed by the speech signal is disentangled into interpretative voice attributes by means of minimizing adversarial loss to make the encoded information mutually independent while preserving the capacity to generate a speech signal from the disentangled codes. During inference for voice conversion, the disentangled voice attributes can be manipulated and the speech signal can be generated accordingly. For experimental evaluation, the proposed method is applied to the task of voice gender conversion using the freely available VCTK dataset. Quantitative measurements of mutual information between the variables of speaker identity and speaker gender show that the proposed architecture can learn gender-independent representation of speakers. Additional measurements of speaker recognition indicate that speaker identity can be recognized accurately from the gender-independent representation. Finally, a subjective experiment conducted on the task of voice gender manipulation shows that the proposed architecture can convert voice gender with very high efficiency and good naturalness.
Fichier principal
Vignette du fichier
entropy-25-00375-v2.pdf (715.49 Ko) Télécharger le fichier
Origine : Fichiers éditeurs autorisés sur une archive ouverte

Dates et versions

hal-04004399 , version 1 (24-02-2023)

Identifiants

Citer

Laurent Benaroya, Nicolas Obin, Axel Roebel. Manipulating Voice Attributes by Adversarial Learning of Structured Disentangled Representations. Entropy, 2023, 25 (2), ⟨10.3390/e25020375⟩. ⟨hal-04004399⟩
47 Consultations
100 Téléchargements

Altmetric

Partager

Gmail Facebook X LinkedIn More