Taking Advantage of Emergent Properties of a Transformer for Music Representation Learning
Résumé
Self-supervised learning (SSL) has proven effective for music information retrieval (MIR), yet contrastive and equivariant methods excel on different tasks: contrastive learning on tagging, equivariant learning on structured predictions. We propose a multi-class-token multitask model (MT2) using multiple class tokens to optimize both contrastive and equivariant pretext tasks simultaneously, taking advantage of the emergent properties of a transformer. Averaging class tokens improves performance across diverse MIR tasks, while sequence tokens capture local musical information using emergent properties in a transformer. Our model achieves competitive results with far fewer parameters than existing masked modeling approaches.
| Origine | Fichiers produits par l'(les) auteur(s) |
|---|---|
| Licence |