Retrieving Speaker Information from Personalized Acoustic Models for Speech Recognition - Archive ouverte HAL Access content directly
Conference Papers Year : 2022

Retrieving Speaker Information from Personalized Acoustic Models for Speech Recognition

Abstract

The widespread of powerful personal devices capable of collecting voice of their users has opened the opportunity to build speaker adapted speech recognition system (ASR) or to participate to collaborative learning of ASR. In both cases, personalized acoustic models (AM), i.e. fine-tuned AM with specific speaker data, can be built. A question that naturally arises is whether the dissemination of personalized acoustic models can leak personal information. In this paper, we show that it is possible to retrieve the gender of the speaker, but also his identity, by just exploiting the weight matrix changes of a neural acoustic model locally adapted to this speaker. Incidentally we observe phenomena that may be useful towards explainability of deep neural networks in the context of speech processing. Gender can be identified almost surely using only the first layers and speaker verification performs well when using middle-up layers. Our experimental study on the TED-LIUM 3 dataset with HMM/TDNN models shows an accuracy of 95% for gender detection, and an Equal Error Rate of 9.07% for a speaker verification task by only exploiting the weights from personalized models that could be exchanged instead of user data.
Fichier principal
Vignette du fichier
ICASSP_2022_SpeakerAnalysisInfoPrivacyVF.pdf (675.48 Ko) Télécharger le fichier
Origin : Files produced by the author(s)

Dates and versions

hal-03539741 , version 1 (22-01-2022)

Identifiers

  • HAL Id : hal-03539741 , version 1

Cite

Salima Mdhaffar, Jean-François Bonastre, Marc Tommasi, Natalia Tomashenko, Yannick Estève. Retrieving Speaker Information from Personalized Acoustic Models for Speech Recognition. IEEE ICASSP 2022, 2022, Singapour, Singapore. ⟨hal-03539741⟩
94 View
113 Download

Share

Gmail Facebook X LinkedIn More