Variable-rate hierarchical CPC leads to acoustic unit discovery in speech - Archive ouverte HAL Accéder directement au contenu
Communication Dans Un Congrès Année : 2022

Variable-rate hierarchical CPC leads to acoustic unit discovery in speech

Résumé

The success of deep learning comes from its ability to capture the hierarchical structure of data by learning high-level representations defined in terms of low-level ones. In this paper we explore self-supervised learning of hierarchical representations of speech by applying multiple levels of Contrastive Predictive Coding (CPC). We observe that simply stacking two CPC models does not yield significant improvements over single-level architectures. Inspired by the fact that speech is often described as a sequence of discrete units unevenly distributed in time, we propose a model in which the output of a low-level CPC module is non-uniformly downsampled to directly minimize the loss of a high-level CPC module. The latter is designed to also enforce a prior of separability and discreteness in its representations by enforcing dissimilarity of successive high-level representations through focused negative sampling, and by quantization of the prediction targets. Accounting for the structure of the speech signal improves upon single-level CPC features and enhances the disentanglement of the learned representations, as measured by downstream speech recognition tasks, while resulting in a meaningful segmentation of the signal that closely resembles phone boundaries.
Fichier principal
Vignette du fichier
NeurIPS-2022-variable-rate-hierarchical-cpc-leads-to-acoustic-unit-discovery-in-speech-Paper-Conference.pdf (285.86 Ko) Télécharger le fichier
Origine : Fichiers éditeurs autorisés sur une archive ouverte
Licence : Copyright (Tous droits réservés)

Dates et versions

hal-04093636 , version 1 (10-05-2023)

Identifiants

Citer

Santiago Cuervo, Adrian Łańcucki, Ricard Marxer, Paweł Rychlikowski, Jan Chorowski. Variable-rate hierarchical CPC leads to acoustic unit discovery in speech. Advances in Neural Information Processing Systems 35 (NeurIPS 2022), Nov 2022, New Orleans, United States. pp.34995-35006. ⟨hal-04093636⟩
7 Consultations
10 Téléchargements

Altmetric

Partager

Gmail Facebook X LinkedIn More