Name
Abstract | ||
|---|---|---|
Data integration is a technique used to combine different sources of data together to provide an unified view among them. MOMIS[1] is an open-source data integration framework developed by the DBGroup1. The goal of our work is to make MOMIS be able to scale-out as the input data sources increase without introducing noticeable performance penalty. In particular, we present a full outer join method capable to efficiently integrate multiple sources at the same time by using data streams and provenance information. To evaluate the scalability of this innovative approach, we developed a join engine employing a distributed data processing framework. Our solution is able to process input data sources in the form of continuous stream, execute the join operation on-the-fly and produce outputs as soon as they are generated. In this way, the join can return partial results before the input streams have been completely received or processed optimizing the entire execution. Encouraging results of adopting the proposed approach on real datasets closes the paper. |
Year | DOI | Venue |
|---|---|---|
2017 | 10.1109/HPCS.2017.23 | 2017 International Conference on High Performance Computing & Simulation (HPCS) |
Keywords | Field | DocType |
Stream processing,full outer join,distributed process | Data integration,Data stream mining,Data processing,Computer science,Distributed database,Stream processing,Open source software,Database,Scalability | Conference |
ISBN | Citations | PageRank |
978-1-5386-3251-2 | 0 | 0.34 |
References | Authors | |
14 | 4 | |
Name | Order | Citations | PageRank |
|---|---|---|---|
| Song Zhu | 1 | 12 | 3.18 |
| Giuseppe Fiameni | 2 | 0 | 0.34 |
| Giovanni Simonini | 3 | 31 | 11.55 |
| Sonia Bergamaschi | 4 | 1240 | 297.26 |