Paper Info
Title
An Edge-Set Based Large Scale Graph Processing System
Abstract
Next generation analytics will be all about graphs, though performance has been a fundamental challenge for large scale graph processing. In this paper, we present an industrial graph processing engine for exploring various large scale linked data, which exhibits superior performance due to the several innovations. This engine organizes a graph as a set of edge-sets, compatible with the traditional edge-centric sharding for graphs, but becomes more amenable for large scale processing. Each time only a portion of the sets are needed for computation and the data access patterns can be highly predictable for prefetch for many graph computing algorithms. Due to the sparsity of large scale graph structure, this engine differentiates logical edge-sets from the edge-sets physically stored on the disk, where multiple logical edge-sets can be organized into a same physical edge-set to increase the data locality. Besides, in contrast to existing solution, the data structures utilized for the physical edge-sets can vary from one to another. Such heterogeneous edge-set representation explores the best graph processing performance according to local data access patterns. We conduct experiments on a representative set of property graphs on multiple platforms, where the proposed system outperform the baseline systems consistently.
Year
Venue
Keywords
2016
2016 IEEE INTERNATIONAL CONFERENCE ON BIG DATA (BIG DATA)
graph, parallel, prefetch, edge-set
Field
DocType
Citations 
Data mining,Locality,Computer science,Linked data,Theoretical computer science,Artificial intelligence,Analytics,Data structure,Graph database,Instruction prefetch,Big data,Data access,Machine learning
Conference
0
PageRank 
References 
Authors
0.34
4
5
Name
Order
Citations
PageRank
Li Zhou121.05
Xia, Yinglong221726.91
Hui Zang3105277.25
Jian Xu400.34
Mingzhen Xia500.34