Leveraging distributed GraphLab for program trace analysis
Résumé
Graph-mining is a class of data-mining problems where programs involve the processing of data modeled as graphs. These applications often exhibit irregular and data-dependent communication patterns, hampering parallelization opportunities on distributed architectures. Many tools and frameworks were created for the scalable processing of graphs but their comparison is non-trivial on distributed architectures as there is no efficiency metrics with respect to distributed resource usage. Considering an in-house use-case, program trace analysis for parallelization optimizations, we study the benefits and limits of a graph-processing framework for a tangible application. The algorithm was implemented using GraphLab and executed on a humble 7-node commodity cluster with input instances up to 40 million vertices and 50 million edges. We propose in this paper an in-depth analysis of the GraphLab system to evaluate its performance and scalability in the context of program trace analysis. The analysis is driven both by traditional and domain-specific metrics and contributes to a better understanding of the system behavior.