Composition of weighted finite transducers in MapReduce - Archive ouverte HAL
Article Dans Une Revue Journal of Big Data Année : 2021

Composition of weighted finite transducers in MapReduce

Résumé

Abstract Weighted finite-state transducers have been shown to be a general and efficient representation in many applications such as text and speech processing, computational biology, and machine learning. The composition of weighted finite-state transducers constitutes a fundamental and common operation between these applications. The NP-hardness of the composition computation problem presents a challenge that leads us to devise efficient algorithms on a large scale when considering more than two transducers. This paper describes a parallel computation of weighted finite transducers composition in MapReduce framework. To the best of our knowledge, this paper is the first to tackle this task using MapReduce methods. First, we analyze the communication cost of this problem using Afrati et al. model. Then, we propose three MapReduce methods based respectively on input alphabet mapping, state mapping, and hybrid mapping. Finally, intensive experiments on a wide range of weighted finite-state transducers are conducted to compare the proposed methods and show their efficiency for large-scale data.

Dates et versions

hal-03515727 , version 1 (06-01-2022)

Identifiants

Citer

Bilal Elghadyry, Faissal Ouardi, Sébastien Verel. Composition of weighted finite transducers in MapReduce. Journal of Big Data, 2021, 8 (1), pp.22. ⟨10.1186/s40537-020-00397-4⟩. ⟨hal-03515727⟩

Collections

UNIV-LITTORAL
38 Consultations
0 Téléchargements

Altmetric

Partager

More