Fractional Hitting Sets for Efficient and Lightweight Genomic Data Sketching - Archive ouverte HAL
Communication Dans Un Congrès Année : 2023

Fractional Hitting Sets for Efficient and Lightweight Genomic Data Sketching

Résumé

The exponential increase in publicly available sequencing data and genomic resources necessitates the development of highly efficient methods for data processing and analysis. Locality-sensitive hashing techniques have successfully transformed large datasets into smaller, more manageable sketches while maintaining comparability using metrics such as Jaccard and containment indices. However, fixed-size sketches encounter difficulties when applied to divergent datasets. Scalable sketching methods, such as Sourmash, provide valuable solutions but still lack resourceefficient, tailored indexing. Our objective is to create lighter sketches with comparable results while enhancing efficiency. We introduce the concept of Fractional Hitting Sets, a generalization of Universal Hitting Sets, which uniformly cover a specified fraction of the k -mer space. In theory and practice, we demonstrate the feasibility of achieving such coverage with simple but highly efficient schemes. By encoding the covered k -mers as super- k -mers, we provide a space-efficient exact representation that also enables optimized comparisons. Our novel tool, SuperSampler, implements this scheme, and experimental results with real bacterial collections closely match our theoretical findings. In comparison to Sourmash, SuperSampler achieves similar outcomes while utilizing an order of magnitude less space and memory and operating several times faster. This highlights the potential of our approach in addressing the challenges presented by the ever-expanding landscape of genomic data. SuperSampler is an open-source software and can be accessed at github.com/TimRouze/supersampler . The data required to reproduce the results presented in this manuscript is available at github.com/TimRouze/Expe_SPSP .
Fichier principal
Vignette du fichier
2023.06.21.545875v1.full.pdf (1.17 Mo) Télécharger le fichier
Origine Fichiers éditeurs autorisés sur une archive ouverte

Dates et versions

hal-04279798 , version 1 (14-11-2023)

Identifiants

Citer

Timothé Rouzé, Igor Martayan, Camille Marchet, Antoine Limasset. Fractional Hitting Sets for Efficient and Lightweight Genomic Data Sketching. 23rd International Workshop on Algorithms in Bioinformatics (WABI 2023, Sep 2023, Houston, United States. ⟨10.4230/LIPIcs.WABI.2023.15⟩. ⟨hal-04279798⟩
215 Consultations
47 Téléchargements

Altmetric

Partager

More