MONTRES : Merge ON-The-Run External Sorting algorithm for large data volumes on SSD based storage systems - Archive ouverte HAL Accéder directement au contenu
Article Dans Une Revue IEEE Transactions on Computers Année : 2017

MONTRES : Merge ON-The-Run External Sorting algorithm for large data volumes on SSD based storage systems

Résumé

External sorting algorithms are commonly used by data-centric applications to sort quantities of data that are larger than the main-memory. Many external sorting algorithms were proposed in state-of-the-art studies to take advantage of SSD performance properties to accelerate the sorting process. In this paper, we demonstrate that unfortunately, many of those algorithms fail to scale when it comes to increasing the dataset size under memory pressure. In order to address this issue, we propose a new sorting algorithm named MONTRES. MONTRES relies on SSD performance model while decreasing the overall number of I/O operations. It does this by reducing the amount of temporary data generated during the sorting process by continuously evicting small values in the final sorted file. MONTRES scales well with growing datasets under memory pressure. We tested MONTRES using several data distributions, different amounts of main-memory workspace and three SSD models. Results showed that MONTRES outperforms state-of-the-art algorithms as it reduces the sorting execution time of TPC-H datasets by more than 30 percent when the file size to main-memory size ratio is high.
Fichier non déposé

Dates et versions

hal-01588600 , version 1 (15-09-2017)

Identifiants

Citer

Arezki Laga, Jalil Boukhobza, Frank Singhoff, Michel Koskas. MONTRES : Merge ON-The-Run External Sorting algorithm for large data volumes on SSD based storage systems. IEEE Transactions on Computers, 2017, 66 (10), pp.1689 - 1702. ⟨10.1109/TC.2017.2706678⟩. ⟨hal-01588600⟩
271 Consultations
0 Téléchargements

Altmetric

Partager

Gmail Facebook X LinkedIn More