A Multi-partitioning Approach to Building Fast and Accurate Counting Bloom Filters - Archive ouverte HAL
Communication Dans Un Congrès Année : 2013

A Multi-partitioning Approach to Building Fast and Accurate Counting Bloom Filters

Résumé

Bloom filters are space-efficient data structures for fast set membership queries. Counting Bloom Filters (CBFs) extend Bloom filters by allowing insertions and deletions to support dynamic sets. The performance of CBFs is critical for various applications and systems. This paper presents a novel approach to building a fast and accurate data structure called Multiple-Partitioned Counting Bloom Filter (MPCBF) that addresses large-scale data processing challenges. MPCBF is based on two ideas: reducing the number of memory accesses from k (for k hash functions) in the standard CBF to only one memory access in the basic MPCBF-1 case, and a hierarchical structure to improve the false positive rate. We also generalize MPCBF-1 to MPCBF-g to accommodate up to g memory accesses. Our simulation and implementation in MapReduce show that MPCBF outperforms the standard CBF in terms of speed and accuracy. Compared to CBF, at the same memory consumption, MPCBF significantly reduces the false positive rate by an order of magnitude, with a reduction of processing overhead by up to 85.9%.
Fichier non déposé

Dates et versions

hal-00945210 , version 1 (11-02-2014)

Identifiants

  • HAL Id : hal-00945210 , version 1

Citer

Kun Huang, Dafang Zhang, Gaogang Xie, Salamatian Kavé, Alex Liu, et al.. A Multi-partitioning Approach to Building Fast and Accurate Counting Bloom Filters. 2013 IEEE 27th International Symposium on Parallel & Distributed Processing (IPDPS),, May 2013, Boston, United States. pp.1159-1170. ⟨hal-00945210⟩
79 Consultations
0 Téléchargements

Partager

More