TextBenDS: a generic Textual data Benchmark for Distributed Systems - Archive ouverte HAL Accéder directement au contenu
Article Dans Une Revue Information Systems Frontiers Année : 2021

TextBenDS: a generic Textual data Benchmark for Distributed Systems

Résumé

Extracting top-k keywords and documents using weighting schemes are popular techniques employed in text mining and machine learning for different analysis and retrieval tasks. The weights are usually computed in the data preprocessing step, as they are costly to update and keep track of all the modifications performed on the dataset. Furthermore, computation errors are introduced when analyzing only subsets of the dataset. Therefore, in a Big Data context, it is crucial to lower the runtime of computing weighting schemes, without hindering the analysis process and the accuracy of the machine learning algorithms. To address this requirement for the task of top-k keywords and documents, it is customary to design benchmarks that compare weighting schemes within various configurations of distributed frameworks and database management systems. Thus, we propose a generic document-oriented benchmark for storing textual data and constructing weighting schemes (TextBenDS). Our benchmark offers a generic data model designed with a multidimensional approach for storing text documents. We also propose using aggregation queries with various complexities and selectivities for constructing term weighting schemes, that are utilized in extracting top-k keywords and documents. We evaluate the computing performance of the queries on several distributed environments set within the Apache Hadoop ecosystem. Our experimental results provide interesting insights. As an example, MongoDB proves to have the best overall performance, while Spark's execution time remains almost the same, regardless of the weighting schemes.
Fichier principal
Vignette du fichier
textbends.pdf (719.49 Ko) Télécharger le fichier
svglov3.clo (3.72 Ko) Télécharger le fichier
Origine : Fichiers produits par l'(les) auteur(s)

Dates et versions

hal-02476758 , version 1 (11-08-2021)

Licence

Paternité

Identifiants

Citer

Ciprian-Octavian Truica, Elena Apostol, Jérôme Darmont, Ira Assent. TextBenDS: a generic Textual data Benchmark for Distributed Systems. Information Systems Frontiers, 2021, Breakthroughs on Cross-Cutting Data Management, Data Analytics and Applied Data Science, 23, pp.81-100. ⟨10.1007/s10796-020-09999-y⟩. ⟨hal-02476758⟩
389 Consultations
70 Téléchargements

Altmetric

Partager

Gmail Facebook X LinkedIn More