LSH SimilarityJoin pattern in FastFlow - Archive ouverte HAL Accéder directement au contenu
Communication Dans Un Congrès Année : 2023

LSH SimilarityJoin pattern in FastFlow

Résumé

Similarity joins are recognized to be among the most used data processing and analysis operations. In this work we introduce a C++-based high-level parallel pattern implemented on top of FastFlow Building Blocks to provide the programmer with ready-to-use similarity joins computations. The SimilarityJoin pattern is implemented according to the MapReduce paradigm enriched with Locality Sensitive Hashing (LSH) to optimize the whole com- putation. The new parallel pattern can be used with any C++ serializable data structure and executed on shared- and distributed-memory machines. We present some experimental validation of the proposed solution on two dif- ferent clusters using the original hand-tuned Hadoop implementation of the LSH-based similarity join algorithms as a reference baseline.
Fichier non déposé

Dates et versions

hal-04575238 , version 1 (14-05-2024)

Identifiants

  • HAL Id : hal-04575238 , version 1

Citer

Nicolò Tonci, Sébastien Rivault, Mostafa Bamha, Sophie Robert, Sébastien Limet, et al.. LSH SimilarityJoin pattern in FastFlow. 16TH INTERNATIONAL SYMPOSIUM ON HIGH-LEVEL PARALLEL PROGRAMMING AND APPLICATIONS, Jun 2023, Cluj-Napoca, Romania. ⟨hal-04575238⟩
6 Consultations
0 Téléchargements

Partager

Gmail Mastodon Facebook X LinkedIn More