Investigating self-supervised speech models' ability to classify animal vocalizations: The case of gibbon's vocal signatures - Laboratoire d'Informatique et Systèmes
Communication Dans Un Congrès Année : 2024

Investigating self-supervised speech models' ability to classify animal vocalizations: The case of gibbon's vocal signatures

Résumé

With the advent of pre-trained self-supervised learning (SSL) models, speech processing research is showing increasing interest towards disentanglement and explainability. Amongst other methods, probing speech classifiers has emerged as a promising approach to gain new insights into SSL models out-of-domain performances. We explore knowledge transfer capabilities of pre-trained speech models with vocalizations from the closest living relatives of humans: non-human primates. We focus on classifying the identity of northern grey gibbons (Hylobates funereus) from their calls with probing and layer-wise analysis of state-of-the-art SSL speech models compared to pre-trained bird species classifiers and audio taggers. By testing the reliance of said models on background noise and timewise information, as well as performance variations across layers, we propose a new understanding of the mechanisms underlying speech models efficacy as bioacoustic tools.
Fichier principal
Vignette du fichier
Jules___Interspeech_2024-6.pdf (2.37 Mo) Télécharger le fichier
Origine Fichiers produits par l'(les) auteur(s)

Dates et versions

hal-04693119 , version 1 (12-09-2024)

Identifiants

Citer

Jules Cauzinille, Benoît Favre, Ricard Marxer, Dena Clink, Abdul Hamid Ahmad, et al.. Investigating self-supervised speech models' ability to classify animal vocalizations: The case of gibbon's vocal signatures. Interspeech 2024, Sep 2024, Kos / Greece, Greece. pp.132-136, ⟨10.21437/Interspeech.2024-1096⟩. ⟨hal-04693119⟩
17 Consultations
24 Téléchargements

Altmetric

Partager

More