LSH SimilarityJoin pattern in FastFlow - Archive ouverte HAL Access content directly
Conference Papers Year : 2023

LSH SimilarityJoin pattern in FastFlow

Abstract

Similarity joins are recognized to be among the most used data processing and analysis operations. In this work we introduce a C++-based high-level parallel pattern implemented on top of FastFlow Building Blocks to provide the programmer with ready-to-use similarity joins computations. The SimilarityJoin pattern is implemented according to the MapReduce paradigm enriched with Locality Sensitive Hashing (LSH) to optimize the whole com- putation. The new parallel pattern can be used with any C++ serializable data structure and executed on shared- and distributed-memory machines. We present some experimental validation of the proposed solution on two dif- ferent clusters using the original hand-tuned Hadoop implementation of the LSH-based similarity join algorithms as a reference baseline.
No file

Dates and versions

hal-04575238 , version 1 (14-05-2024)

Identifiers

  • HAL Id : hal-04575238 , version 1

Cite

Nicolò Tonci, Sébastien Rivault, Mostafa Bamha, Sophie Robert, Sébastien Limet, et al.. LSH SimilarityJoin pattern in FastFlow. 16TH INTERNATIONAL SYMPOSIUM ON HIGH-LEVEL PARALLEL PROGRAMMING AND APPLICATIONS, Jun 2023, Cluj-Napoca, Romania. ⟨hal-04575238⟩
0 View
0 Download

Share

Gmail Mastodon Facebook X LinkedIn More