Topological Data Analysis for Speech Processing - Archive ouverte HAL Accéder directement au contenu
Article Dans Une Revue Proceedings of the Annual Conference of the International Speech Communication Association, INTERSPEECH Année : 2023

Topological Data Analysis for Speech Processing

Résumé

We apply topological data analysis (TDA) to speech classification problems and to the introspection of a pretrained speech model, HuBERT. To this end, we introduce a number of topological and algebraic features derived from Transformer attention maps and embeddings. We show that a simple linear classifier built on top of such features outperforms a fine-tuned classification head. We achieve an improvement of about 9% accuracy and 5% ERR on two common datasets; on CREMA-D, the proposed feature set reaches a new state of the art performance with accuracy 80.155. We also show that topological features are able to reveal functional roles of speech Transformer heads; e.g., we find the heads capable to distinguish between pairs of sample sources (natural/synthetic) or voices without any downstream fine-tuning. Our results demonstrate that TDA is a promising new approach for speech analysis, especially for tasks that require structural prediction.
Fichier principal
Vignette du fichier
TDAinterspeech23.pdf (866.49 Ko) Télécharger le fichier
Origine : Fichiers éditeurs autorisés sur une archive ouverte

Dates et versions

hal-04250733 , version 1 (20-10-2023)

Identifiants

Citer

Eduard Tulchinskii, Kristian Kuznetsov, Daniil Cherniavskii, Serguei Barannikov, Sergey Nikolenko, et al.. Topological Data Analysis for Speech Processing. Proceedings of the Annual Conference of the International Speech Communication Association, INTERSPEECH, 2023, pp.311-315. ⟨10.21437/Interspeech.2023-1861⟩. ⟨hal-04250733⟩
10 Consultations
24 Téléchargements

Altmetric

Partager

Gmail Facebook X LinkedIn More