Supervised classification of end-of-lines in clinical text with no manual annotation - Archive ouverte HAL Accéder directement au contenu
Communication Dans Un Congrès Année : 2016

Supervised classification of end-of-lines in clinical text with no manual annotation

Résumé

In some plain text documents, end-of-line marks may or may not mark the boundary of a text unit (e.g., of a paragraph). This vexing problem is likely to impact subsequent natural language processing components, but is seldom addressed in the literature. We propose a method which uses no manual annotation to classify whether end-of-lines must actually be seen as simple spaces (soft line breaks) or as true text unit boundaries. This method, which includes self-training and co-training steps based on token and line length features, achieves 0.943 F-measure on a corpus of short e-books with controlled format, F=0.904 on a random sample of 24 clinical texts with soft line breaks, and F=0.898 on a larger set of mixed clinical texts which may or may not contain soft line breaks, a fairly high value for a method with no manual annotation.
Fichier non déposé

Dates et versions

hal-01831235 , version 1 (05-07-2018)

Identifiants

  • HAL Id : hal-01831235 , version 1

Citer

Pierre Zweigenbaum, Cyril Grouin, Thomas Lavergne. Supervised classification of end-of-lines in clinical text with no manual annotation. Workshop on Building and Evaluating Resources for Biomedical Text Mining, Dec 2016, Osaka, Japan. ⟨hal-01831235⟩
31 Consultations
0 Téléchargements

Partager

Gmail Facebook X LinkedIn More