Normalizing speech transcriptions for Natural Language Processing
Résumé
Researchers in the field of spoken text processing face specific problems, all related to the nature of the data. In particular, spoken texts are full of disfluencies that constitute practical issues for automatic analysis. On the basis of a corpus of almost 500.000 words from the textual data bank of spontaneous spoken French of Valibel (http://www.uclouvain.be/valibel.html), we have especially studied four types of disfluencies: repetition, word fragments, immediate self-correction and the word euh, called "filled pause". In this paper, we show how these four types of disfluencies were automatically preprocessed in texts. The principle we used was to annotate the part of the disfluency called reparandum (according to the terminology in Shriberg 1994), in order to keep only the repair part.