Name
Abstract | ||
|---|---|---|
Kineograph is a distributed system that takes a stream of incoming data to construct a continuously changing graph, which captures the relationships that exist in the data feed. As a computing platform, Kineograph further supports graph-mining algorithms to extract timely insights from the fast-changing graph structure. To accommodate graph-mining algorithms that assume a static underlying graph, Kineograph creates a series of consistent snapshots, using a novel and efficient epoch commit protocol. To keep up with continuous updates on the graph, Kineograph includes an incremental graph-computation engine. We have developed three applications on top of Kineograph to analyze Twitter data: user ranking, approximate shortest paths, and controversial topic detection. For these applications, Kineograph takes a live Twitter data feed and maintains a graph of edges between all users and hashtags. Our evaluation shows that with 40 machines processing 100K tweets per second, Kineograph is able to continuously compute global properties, such as user ranks, with less than 2.5-minute timeliness guarantees. This rate of traffic is more than 10 times the reported peak rate of Twitter as of October 2011. |
Year | DOI | Venue |
|---|---|---|
2012 | 10.1145/2168836.2168846 | EuroSys |
Keywords | Field | DocType |
live twitter data feed,twitter data,data feed,connected world,graph-mining algorithm,reported peak rate,user rank,static underlying graph,incoming data,fast-changing graph structure,user ranking,distributed storage,shortest path,distributed system | Graph,Graph database,Ranking,Commit,Computer science,Distributed data store,Theoretical computer science,Real-time computing,Snapshot (computer storage),Data feed,Distributed computing | Conference |
Citations | PageRank | References |
107 | 4.52 | 24 |
Authors | ||
10 | ||
Name | Order | Citations | PageRank |
|---|---|---|---|
| Raymond Cheng | 1 | 140 | 9.46 |
| Ji Hong | 2 | 107 | 4.52 |
| Aapo Kyrola | 3 | 1049 | 33.52 |
| Youshan Miao | 4 | 196 | 9.97 |
| Xuetian Weng | 5 | 222 | 8.93 |
| Ming Wu | 6 | 901 | 62.61 |
| Fan Yang | 7 | 193 | 9.85 |
| Lidong Zhou | 8 | 2136 | 147.82 |
| Feng Zhao | 9 | 4593 | 455.17 |
| Enhong Chen | 10 | 2106 | 165.57 |