Paper Info
Title
A Lightweight Communication Runtime for Distributed Graph Analytics.
Abstract
Distributed-memory multi-core clusters enable in-memory processing of very large graphs with billions of nodes and edges. Recent distributed graph analytics systems have been built on top of MPI. However, communication in graph applications is very irregular, and each host exchanges different amounts of non-contiguous data with other hosts. MPI does not support such a communication pattern well, and it has limited ability to integrate communication with serialization, deserialization, and graph computation tasks. In this paper, we describe a lightweight communication runtime called LCI that supports a large number of threads on each host and avoids the semantic mismatches between the requirements of graph computations and the communication library in MPI. The implementation of LCI is informed by lessons learnt from two baseline MPI-based implementations. We have successfully integrated LCI with two state-of-the-art graph analytics systems - Gemini and Abelian. LCI improves the latency up to 3.5x for microbenchmarks compared to MPI solutions and improves the end-to-end performance of distributed graph algorithms by up to 2x.
Year
Venue
Field
2018
IPDPS
Synchronization,Serialization,Computer science,Parallel computing,Communications system,Implementation,Thread (computing),Semantics,Benchmark (computing),Computation,Distributed computing
DocType
Citations 
PageRank 
Conference
2
0.36
References 
Authors
0
9
Name
Order
Citations
PageRank
Hoang-Vu Dang19610.04
Roshan Dathathri2797.22
Gurbinder Singh Gill3505.34
Alex Brooks414913.10
Nikoli Dryden5363.58
Andrew Lenharth645619.94
Loc Hoang7312.37
Keshav Pingali83056256.64
M. Snir93984520.82