Text Alignment from Bimodal Mathematical Expression Sources - Archive ouverte HAL Access content directly
Conference Papers Year : 2014

Text Alignment from Bimodal Mathematical Expression Sources

Sofiane Medjkoune
  • Function : Author
  • PersonId : 907995
Harold Mouchère
Christian Viard-Gaudin


In this paper we propose a new approach tomerge mathematical expression recognition results coming fromhandwriting and speech modalities. Using a bimodal descriptionof mathematical expressions allows taking advantage of thecomplementarities between both signals, and can disambiguatesituations were a single modality would not be clear enough. Tocombine the signals coming from both modalities, we propose torepresent them in the same space as a textual description. First,from the handwriting signal, we generate the Nbest mathematicalexpressions; each of them is next translated as different possiblestrings. From the audio signal, an automatic speech recognitionsystem provides a transcript, which is also available as a string.A string comparison algorithm is achieved to select the bestmathematical expressions. This bimodal system is evaluated onreal bimodal data from the HAMEX dataset and the results arecompared to a single modality (handwriting) based system.
Fichier principal
Vignette du fichier
MedjkouneICFHR2014_final.pdf (2.75 Mo) Télécharger le fichier
Origin : Publisher files allowed on an open archive

Dates and versions

hal-01096510 , version 1 (17-12-2014)



Sofiane Medjkoune, Harold Mouchère, Christian Viard-Gaudin, Simon Petitrenaud. Text Alignment from Bimodal Mathematical Expression Sources. 14th International Conference on Frontiers in Handwriting Recognition, Sep 2014, Crete, Greece. pp.205--209, ⟨10.1109/ICFHR.2014.42⟩. ⟨hal-01096510⟩
478 View
71 Download



Gmail Facebook Twitter LinkedIn More