Topological Data Analysis for Speech Processing
Résumé
We apply topological data analysis (TDA) to speech classification problems and to the introspection of a pretrained speech model, HuBERT. To this end, we introduce a number of topological and algebraic features derived from Transformer attention maps and embeddings. We show that a simple linear classifier built on top of such features outperforms a fine-tuned classification head. We achieve an improvement of about 9% accuracy and 5% ERR on two common datasets; on CREMA-D, the proposed feature set reaches a new state of the art performance with accuracy 80.155. We also show that topological features are able to reveal functional roles of speech Transformer heads; e.g., we find the heads capable to distinguish between pairs of sample sources (natural/synthetic) or voices without any downstream fine-tuning. Our results demonstrate that TDA is a promising new approach for speech analysis, especially for tasks that require structural prediction.
Domaines
Intelligence artificielle [cs.AI] Intelligence artificielle [cs.AI] Géométrie algorithmique [cs.CG] Informatique et langage [cs.CL] Vision par ordinateur et reconnaissance de formes [cs.CV] Apprentissage [cs.LG] Réseau de neurones [cs.NE] Topologie algébrique [math.AT] Machine Learning [stat.ML]
Origine : Fichiers éditeurs autorisés sur une archive ouverte