Self-Supervised Learning of Multi-level Audio Representations for Music Segmentation - Archive ouverte HAL Accéder directement au contenu
Article Dans Une Revue IEEE/ACM Transactions on Audio, Speech and Language Processing Année : 2024

Self-Supervised Learning of Multi-level Audio Representations for Music Segmentation

Apprentissage Auto-supervisé de Représentations Audio Multi-niveaux pour la Segmentation Musicale

Résumé

The task of music structure analysis refers to automatically identifying the location and the nature of musical sections within a song. In the supervised scenario, structural annotations generally result from exhaustive data collection processes, which represents one of the main challenges of this task. Moreover, both the subjectivity of music structure and the hierarchical characteristics it exhibits make the obtained structural annotations not fully reliable, in the sense that they do not convey a "universal ground-truth" unlike other tasks in music information retrieval. On the other hand, the quickly growing quantity of available music data has enabled weakly supervised and self-supervised approaches to achieve impressive results on a wide range of music-related problems. In this work, a self-supervised learning method is proposed to learn robust multi-level music representations prior to structural segmentation using contrastive learning. To this end, sets of frames sampled at different levels of detail are used to train a deep neural network in a disentangled manner. The proposed method is evaluated on both flat and multi-level segmentation. We show that each distinct sub-region of the output embeddings can efficiently account for structural similarity at their own targeted level of detail, which ultimately improves performance of downstream flat and multi-level segmentation. Finally, complementary experiments are carried out to study how the obtained representations can be further adapted to specific datasets using a supervised fine-tuning objective in order to facilitate structure retrieval in domains where human annotations remain scarce.
Fichier principal
Vignette du fichier
Buisson.pdf (6.3 Mo) Télécharger le fichier
Origine : Fichiers produits par l'(les) auteur(s)

Dates et versions

hal-04485065 , version 1 (01-03-2024)

Identifiants

Citer

Morgan Buisson, Brian Mcfee, Slim Essid, Hélène Crayencour. Self-Supervised Learning of Multi-level Audio Representations for Music Segmentation. IEEE/ACM Transactions on Audio, Speech and Language Processing, 2024, pp.1-13. ⟨10.1109/TASLP.2024.3379894⟩. ⟨hal-04485065⟩
56 Consultations
35 Téléchargements

Altmetric

Partager

Gmail Facebook X LinkedIn More