Zero-Note Samba: Self-Supervised Beat Tracking - Archive ouverte HAL Accéder directement au contenu
Article Dans Une Revue IEEE/ACM Transactions on Audio, Speech and Language Processing Année : 2023

Zero-Note Samba: Self-Supervised Beat Tracking

Zero-Note Samba : Suivi de pulsation autodidacte

Résumé

Supervised machine learning for music information retrieval requires a large annotated training set, and thus a high cognitive workload. To circumvent this problem, we propose to train deep neural networks to perceive beats in musical recordings despite having little or no access to human annotations. The key idea, which we name "Zero-Note Samba" (ZeroNS), is to train two fully convolutional networks in parallel: the first analyzes the percussive part of a musical piece whilst the second analyzes its non-percussive part. These networks learn a self-supervised pretext task of synchrony prediction (sync-pred), which simulates the ability of musicians to groove together when playing in the same band. Sync-pred encourages the two networks to return similar outputs if the underlying musical parts are synchronized, yet dissimilar outputs if the parts are out of sync. In practice, we obtain the instrumental parts from commercial recordings via an off-the-shelf source separation system: Spleeter. After self-supervised learning with sync-pred, ZeroNS produces a sparse output that resembles a beat detection function. When used in conjunction with a dynamic Bayesian network, ZeroNS surpasses the state of the art in unsupervised beat tracking. Furthermore, fine-tuning ZeroNS to a small set of labeled data (of the order of one to ten songs) matches the performance of a fully supervised network on 96 songs. Lastly, we show that pre-training a supervised model with sync-pred mitigates dataset bias and thus improves cross-dataset generalization, at no extra annotation cost.
Fichier principal
Vignette du fichier
desblancs2022taslp_zero-note-samba.pdf (1.87 Mo) Télécharger le fichier
Origine : Fichiers produits par l'(les) auteur(s)

Dates et versions

hal-03669865 , version 1 (17-05-2022)
hal-03669865 , version 2 (22-07-2023)

Licence

Paternité - Pas d'utilisation commerciale

Identifiants

Citer

Dorian Desblancs, Vincent Lostanlen, Romain Hennequin. Zero-Note Samba: Self-Supervised Beat Tracking. IEEE/ACM Transactions on Audio, Speech and Language Processing, 2023, pp.1-13. ⟨10.1109/TASLP.2023.3297963⟩. ⟨hal-03669865v2⟩
144 Consultations
555 Téléchargements

Altmetric

Partager

Gmail Facebook X LinkedIn More