On the Fly Detection of the Top-k Items in the Distributed Sliding Window Model - Archive ouverte HAL
Communication Dans Un Congrès Année : 2018

On the Fly Detection of the Top-k Items in the Distributed Sliding Window Model

Résumé

This paper presents a new algorithm that detects on the fly the k most frequent items in the sliding window model. This algorithm is distributed among the nodes of the system. It is inspired by a recent and innovative approach, which consists in associating a stochastic value correlated with the item's frequency instead of trying to estimate its number of occurrences. This stochastic value corresponds to the number of consecutive heads in coin flipping until the first tail occurs. The original approach was to retain just the maximum of consecutive heads obtained by an item, since an item that often occurs will have a higher probability of having a high value. While effective for very skewed data distributions, the correlation is not tight enough to robustly distinguish items with comparable frequencies. To address this important issue, we propose to combine the stochastic approach together with a deterministic counting of items. Specifically, in place of keeping the maximum number of consecutive heads obtained by an item, we count the number of times the coin flipping process of an item has exceeded a given threshold. This threshold is defined by combining theoretical results in leader election and coupon collector problems. Results on simulated data show how impressive is the detection of the top-k items in a large range of distributions.
Fichier principal
Vignette du fichier
NCA_2018.pdf (591.6 Ko) Télécharger le fichier
Origine Fichiers produits par l'(les) auteur(s)
Loading...

Dates et versions

hal-01888298 , version 1 (08-10-2018)

Identifiants

Citer

Emmanuelle Anceaume, Yann Busnel, Vasile Cazacu. On the Fly Detection of the Top-k Items in the Distributed Sliding Window Model. NCA 2018 - 17th IEEE International Symposium on Network Computing and Applications, IEEE, Nov 2018, Boston, United States. pp.1-8, ⟨10.1109/NCA.2018.8548097⟩. ⟨hal-01888298⟩
202 Consultations
573 Téléchargements

Altmetric

Partager

More