Book Sections Year : 2018

Handling Pregel's limits in big graphs processing in the presence of high degree vertices

Abstract

Even if specialized distributed graph processing systems such as Pregel scale better than pure MapReduce programs, in graph processing, by reducing disk I/O for iterative algorithms while offering an easy programming model using "think like vertex" paradigm, large scale graphs processing is still challenging in the presence of high degree vertices: Communication and load imbalance among processing nodes can have disastrous effects on performance. In this article, we introduce a scalable MapReduce graph partitioning approach for high degree vertices using master/slave partitioning. This partitioning makes Pregel-like systems, in graph processing, scalable and insensitive to the effects of high degree vertices while guaranteeing perfect balancing properties of communication and computation during all the stages of big graphs processing. A cost model and performance analysis are given to show the effectiveness and the scalability of our graph partitioning approach in large scale systems.

Dates and versions

hal-01802435 , version 1 (29-05-2018)

Identifiers

Cite

Mohamad Al Hajj Hassan, Mostafa Bamha. Handling Pregel's limits in big graphs processing in the presence of high degree vertices. Applications of Big Data Analytics: Trends, Issues, and Challenges, 2018, 9783319764719. ⟨10.1007/978-3-319-76472-6⟩. ⟨hal-01802435⟩
113 View
0 Download

Altmetric

Share

More