Name
Abstract | ||
|---|---|---|
A distributed join is a fundamental operation for processing massive datasets in parallel. Unfortunately, computing an equi-join over such datasets is very resource-intensive, even when done in parallel. Given this cost, the equi-join operator becomes a natural candidate for optimization using approximation techniques, which allow users to trade accuracy for latency. Finding the right approximation technique for joins, however, is a challenging task. Sampling, in particular, cannot be directly used in joins; naïvely performing a join over a sample of the dataset will not preserve statistical properties of the query result.
To address this problem, we introduce ApproxJoin. We interweave Bloom filter sketching and stratified sampling with the join computation in a new operator that preserves statistical properties of an aggregation over the join output. ApproxJoin leverages Bloom filters to avoid shuffling non-joinable data items around the network, and then applies stratified sampling to obtain a representative sample of the join output. We implemented ApproxJoin in Apache Spark, and evaluated it using microbenchmarks and real-world workloads. Our evaluation shows that ApproxJoin scales well and significantly reduces data movement, without sacrificing tight error bounds on the accuracy of the final results. ApproxJoin achieves a speedup of up to 9x over unmodified Spark-based joins with the same sampling ratio. Furthermore, the speedup is accompanied by a significant reduction in the shuffled data volume, which is up to 82x less than unmodified Spark-based joins.
|
Year | DOI | Venue |
|---|---|---|
2018 | 10.1145/3267809.3267834 | SoCC '18: ACM Symposium on Cloud Computing
Carlsbad
CA
USA
October, 2018 |
Keywords | Field | DocType |
Approximate join processing, multi-way joins, stratified sampling, approximate computing and distributed systems | Bloom filter,Joins,Spark (mathematics),Computer science,Parallel computing,Real-time computing,Shuffling,Operator (computer programming),Sampling (statistics),Speedup,Computation | Conference |
ISBN | Citations | PageRank |
978-1-4503-6011-1 | 1 | 0.35 |
References | Authors | |
32 | 7 | |
Name | Order | Citations | PageRank |
|---|---|---|---|
| Le Quoc, D. | 1 | 55 | 8.21 |
| Istemi Ekin Akkus | 2 | 68 | 6.96 |
| Pramod Bhatotia | 3 | 414 | 28.94 |
| Spyros Blanas | 4 | 550 | 29.56 |
| Ruichuan Chen | 5 | 205 | 18.95 |
| Christof Fetzer | 6 | 2429 | 172.89 |
| Thorsten Strufe | 7 | 846 | 80.61 |