Deep audio embeddings for vocalisation clustering - Archive ouverte HAL Accéder directement au contenu
Article Dans Une Revue PLoS ONE Année : 2023

Deep audio embeddings for vocalisation clustering

Paul Best
Ricard Marxer

Résumé

The study of non-human animals' communication systems generally relies on the transcription of vocal sequences using a finite set of discrete units. This set is referred to as a vocal repertoire, which is specific to a species or a subgroup of a species. When conducted by human experts, the formal description of vocal repertoires can be laborious and/or biased. This motivates computerised assistance for this procedure, for which machine learning algorithms represent a good opportunity. Unsupervised clustering algorithms are suited for grouping close points together, provided a relevant representation. This paper therefore studies a new method for encoding vocalisations, allowing for automatic clustering to alleviate vocal repertoire characterisation. Borrowing from deep representation learning, we use a convolutional auto-encoder network to learn an abstract representation of vocalisations. We report on the quality of the learnt representation, as well as of state of the art methods, by quantifying their agreement with expert labelled vocalisation types from 8 datasets of other studies across 6 species (birds and marine mammals). With this benchmark, we demonstrate that using auto-encoders improves the relevance of vocalisation representation which serves repertoire characterisation using a very limited number of settings. We also publish a Python package for the bioacoustic community to train their own vocalisation autoencoders or use a pretrained encoder to browse vocal repertoires and ease unit wise annotation.
Fichier principal
Vignette du fichier
journal.pone.0283396.pdf (2.65 Mo) Télécharger le fichier
Origine : Publication financée par une institution

Dates et versions

hal-04194226 , version 1 (02-09-2023)

Identifiants

Citer

Paul Best, Sébastien Paris, Hervé Glotin, Ricard Marxer. Deep audio embeddings for vocalisation clustering. PLoS ONE, 2023, 18 (7), pp.e0283396. ⟨10.1371/journal.pone.0283396⟩. ⟨hal-04194226⟩
4 Consultations
14 Téléchargements

Altmetric

Partager

Gmail Facebook X LinkedIn More