Streaming METIS Partitioning
Abstract
The proliferation in size of actual graph datasets impels the use of distributed graph processing frameworks which in turn, should consider a good partitioning of the graph dataset in order to see their performances enhanced. In this paper, we focus on a well known heuristic for graph partitioning named METIS, an offline method giving high quality partitions but unsuitable for processing large graphs due to the offline setting. A recently proposed alternative is the streaming partitioning heuristics aiming to alleviate the computational resources constraints when dealing with large graphs. In considering this matter, we propose a new partitioning method that benefits from the accuracy of METIS and the lightness of the streaming setting. This work introduces the Streaming METIS Partitioning method (SMP) which is an online counterpart of METIS, a fast and well known multilevel heuristic for graph partitioning. We show in a complexity analysis that SMP has a lower time complexity compared to METIS, which is confirmed by conducted experiments. Moreover, we show that SMP yields competitive results to its offline counterpart METIS, especially when it is run on a Depth First Search streaming order. Also, when compared to other online competitors, SMP is the best performing heuristic giving partitions with minimal edge cut.