From HEAR to GEAR: Generative Evaluation of Audio Representations - Archive ouverte HAL Accéder directement au contenu
Article Dans Une Revue Proceedings of Machine Learning Research Année : 2023

From HEAR to GEAR: Generative Evaluation of Audio Representations

Résumé

The "Holistic Evaluation of Audio Representations" (HEAR) is an emerging research program towards statistical models that can transfer to diverse machine listening tasks. The originality of HEAR is to conduct a fair, "apples-to-apples" comparison of many deep learning models over many datasets, resulting in multitask evaluation metrics that are readily interpretable by practitioners. On the flip side, this comparison incurs a neural architecture search: as such, it is not directly interpretable in terms of audio signal processing. In this paper, we propose a complementary viewpoint on the HEAR benchmark, which we name GEAR: Generative Evaluation of Audio Representations. The key idea behind GEAR is to generate a dataset of sounds with few independent factors of variability, analyze it with HEAR embeddings, and visualize it with an unsupervised manifold learning algorithm. Visual inspection reveals stark contrasts in the global structure of the nearest-neighbor graphs associated to logmelspec, Open-L 3 , BYOL, CREPE, wav2vec2, GURA, and YAMNet. Although GEAR currently lacks mathematical refinement, we intend it as a proof of concept to show the potential of parametric audio synthesis in general-purpose machine listening research.
Fichier principal
Vignette du fichier
lostanlen2022hear.pdf (11.45 Mo) Télécharger le fichier
Origine : Fichiers éditeurs autorisés sur une archive ouverte

Dates et versions

hal-03979667 , version 1 (08-02-2023)

Identifiants

  • HAL Id : hal-03979667 , version 1

Citer

Vincent Lostanlen, Lingyao Yan, Xianyi Yang. From HEAR to GEAR: Generative Evaluation of Audio Representations. Proceedings of Machine Learning Research, 2023, 166, pp.48-64. ⟨hal-03979667⟩
102 Consultations
22 Téléchargements

Partager

Gmail Facebook X LinkedIn More