Restoration of Arabic Diacritics Using a Multilevel Statistical Model - Archive ouverte HAL
Conference Papers Year : 2015

Restoration of Arabic Diacritics Using a Multilevel Statistical Model

Abstract

Arabic texts are generally written without diacritics. This is the case for instance in newspapers, contemporary books, etc., which makes automatic processing of Arabic texts more difficult. When diacritical signs are present, Arabic script provides more information about the meanings of words and their pronunciation. Vocalization of Arabic texts is a complex task which may involve morphological, syntactic and semantic text processing.In this paper, we present a new approach to restore Arabic diacritics using a statistical language model and dynamic programming. Our system is based on two models: a bi-gram-based model which is first used for vocalization and a 4-gram character-based model which is then used to handle the words that remain non vocalized (OOV words). Moreover, smoothing methods are used in order to handle the problem of unseen words. The optimal vocalized word sequence is selected using the Viterbi algorithm from Dynamic Programming.Our approach represents an important contribution to the improvement of the performance of automatic Arabic vocalization. We have compared our results with some of the most efficient up-to-date vocalization systems; the experimental results show the high quality of our approach.
Fichier principal
Vignette du fichier
339159_1_En_15_Chapter.pdf (611.85 Ko) Télécharger le fichier
Origin Files produced by the author(s)
Loading...

Dates and versions

hal-01789976 , version 1 (11-05-2018)

Licence

Identifiers

Cite

Mohamed Seghir Hadj Ameur, Youcef Moulahoum, Ahmed Guessoum. Restoration of Arabic Diacritics Using a Multilevel Statistical Model. 5th International Conference on Computer Science and Its Applications (CIIA), May 2015, Saida, Algeria. pp.181-192, ⟨10.1007/978-3-319-19578-0_15⟩. ⟨hal-01789976⟩
336 View
485 Download

Altmetric

Share

More