Dynamic Multi-Probe LSH: an I/O Efficient Index Structure for Approximate Nearest Neighbor Search - Archive ouverte HAL
Communication Dans Un Congrès Année : 2013

Dynamic Multi-Probe LSH: an I/O Efficient Index Structure for Approximate Nearest Neighbor Search

Résumé

Locality-Sensitive Hashing (LSH) is widely used to solve approximate nearest neighbor search problems in high-dimensional spaces. The basic idea is to map the “nearby” objects into a same hash bucket with high probability. A significant drawback is that LSH requires a large number of hash tables to achieve good search quality. Multi-probe LSH was proposed to reduce the number of hash tables by looking up multiple buckets in each table. While optimized for a main memory database, it is not optimal when multi-dimensional vectors are stored in a secondary storage, because the probed buckets may be randomly distributed in different physical pages. In order to optimize the I/O efficiency, we propose a new method called Dynamic Multi-probe LSH which groups small hash buckets into a single bucket by dynamically increasing the number of hash functions during the index construction. Experimental results show that our method is significantly more I/O efficient.

Dates et versions

hal-00834497 , version 1 (15-06-2013)

Identifiants

Citer

Shaoyi Yin, Mehdi Badr, Dan Vodislav. Dynamic Multi-Probe LSH: an I/O Efficient Index Structure for Approximate Nearest Neighbor Search. 24th International Conference on Database and Expert Systems Applications (DEXA 2013), Aug 2013, Prague, Czech Republic. pp.48-62, ⟨10.1007/978-3-642-40285-2_7⟩. ⟨hal-00834497⟩
188 Consultations
0 Téléchargements

Altmetric

Partager

More