Restoration of Arabic Diacritics Using a Multilevel Statistical Model - Archive ouverte HAL
Communication Dans Un Congrès Année : 2015

Restoration of Arabic Diacritics Using a Multilevel Statistical Model

Résumé

Arabic texts are generally written without diacritics. This is the case for instance in newspapers, contemporary books, etc., which makes automatic processing of Arabic texts more difficult. When diacritical signs are present, Arabic script provides more information about the meanings of words and their pronunciation. Vocalization of Arabic texts is a complex task which may involve morphological, syntactic and semantic text processing.In this paper, we present a new approach to restore Arabic diacritics using a statistical language model and dynamic programming. Our system is based on two models: a bi-gram-based model which is first used for vocalization and a 4-gram character-based model which is then used to handle the words that remain non vocalized (OOV words). Moreover, smoothing methods are used in order to handle the problem of unseen words. The optimal vocalized word sequence is selected using the Viterbi algorithm from Dynamic Programming.Our approach represents an important contribution to the improvement of the performance of automatic Arabic vocalization. We have compared our results with some of the most efficient up-to-date vocalization systems; the experimental results show the high quality of our approach.
Fichier principal
Vignette du fichier
339159_1_En_15_Chapter.pdf (611.85 Ko) Télécharger le fichier
Origine Fichiers produits par l'(les) auteur(s)
Loading...

Dates et versions

hal-01789976 , version 1 (11-05-2018)

Licence

Identifiants

Citer

Mohamed Seghir Hadj Ameur, Youcef Moulahoum, Ahmed Guessoum. Restoration of Arabic Diacritics Using a Multilevel Statistical Model. 5th International Conference on Computer Science and Its Applications (CIIA), May 2015, Saida, Algeria. pp.181-192, ⟨10.1007/978-3-319-19578-0_15⟩. ⟨hal-01789976⟩
340 Consultations
485 Téléchargements

Altmetric

Partager

More