Real-Time Audio-to-Score Alignment of Singing Voice Based on Melody and Lyric Information - Archive ouverte HAL Access content directly
Conference Papers Year : 2015

Real-Time Audio-to-Score Alignment of Singing Voice Based on Melody and Lyric Information

Abstract

Singing voice is specific in music: a vocal performance conveys both music (melody/pitch) and lyrics (text/phoneme) content. This paper aims at exploiting the advantages of melody and lyric information for real-time audio-to-score alignment of singing voice. First, lyrics are added as a separate observation stream into a template-based hidden semi-Markov model (HSMM), whose observation model is based on the construction of vowel templates. Second, early and late fusion of melody and lyric information are processed during real-time audio-to-score alignment. An experiment conducted with two professional singers (male/female) shows that the performance of a lyrics-based system is comparable to that of melody-based score following systems. Furthermore, late fusion of melody and lyric information substantially improves the alignment performance. Finally, maximum a posteriori adaptation (MAP) of the vowel templates from one singer to the other suggests that lyric information can be efficiently used for any singer.
Fichier principal
Vignette du fichier
index.pdf (330.53 Ko) Télécharger le fichier
Origin : Files produced by the author(s)

Dates and versions

hal-01164550 , version 1 (17-06-2015)

Identifiers

  • HAL Id : hal-01164550 , version 1

Cite

Rong Gong, Philippe Cuvillier, Nicolas Obin, Arshia Cont. Real-Time Audio-to-Score Alignment of Singing Voice Based on Melody and Lyric Information. Interspeech, Sep 2015, Dresde, Germany. ⟨hal-01164550⟩
408 View
763 Download

Share

Gmail Facebook X LinkedIn More