CONTENT BASED SINGING VOICE SOURCE SEPARATION VIA STRONG CONDITIONING USING ALIGNED PHONEMES - Archive ouverte HAL Access content directly
Conference Papers Year :

CONTENT BASED SINGING VOICE SOURCE SEPARATION VIA STRONG CONDITIONING USING ALIGNED PHONEMES

(1, 2) , (3, 4, 5)
1
2
3
4
5

Abstract

Informed source separation has recently gained renewed interest with the introduction of neural networks and the availability of large multitrack datasets containing both the mixture and the separated sources. These approaches use prior information about the target source to improve separation. Historically, Music Information Retrieval researchers have focused primarily on scoreinformed source separation, but more recent approaches explore lyrics-informed source separation. However, because of the lack of multitrack datasets with time-aligned lyrics, models use weak conditioning with non-aligned lyrics. In this paper, we present a multimodal multitrack dataset with lyrics aligned in time at the word level with phonetic information as well as explore strong conditioning using the aligned phonemes. Our model follows a U-Net architecture and takes as input both the magnitude spectrogram of a musical mixture and a matrix with aligned phonetic information. The phoneme matrix is embedded to obtain the parameters that control Feature-wise Linear Modulation (FiLM) layers. These layers condition the U-Net feature maps to adapt the separation process to the presence of different phonemes via affine transformations. We show that phoneme conditioning can be successfully applied to improve singing voice source separation.
Fichier principal
Vignette du fichier
2008.02070.pdf (1022.21 Ko) Télécharger le fichier
Origin : Files produced by the author(s)

Dates and versions

hal-03200161 , version 1 (16-04-2021)

Identifiers

  • HAL Id : hal-03200161 , version 1

Cite

Gabriel Meseguer-Brocal, Geoffroy Peeters. CONTENT BASED SINGING VOICE SOURCE SEPARATION VIA STRONG CONDITIONING USING ALIGNED PHONEMES. 21st International Society for Music Information Retrieval Conference, Oct 2020, Montréal (virtual), Canada. ⟨hal-03200161⟩
42 View
69 Download

Share

Gmail Facebook Twitter LinkedIn More