Name
Abstract | ||
|---|---|---|
The ability to collect and analyze large amounts of data is a growing problem within the scientific community. The growing gap between data and users calls for innovative tools that address the challenges faced by big data volume, velocity and variety. Numerous tools exist that allow users to store, query and index these massive quantities of data. Each storage or database engine comes with the promise of dealing with complex data. Scientists and engineers who wish to use these systems often quickly find that there is no single technology that offers a panacea to the complexity of information. When using multiple technologies, however, there is significant trouble in designing the movement of information between storage and database engines to support an end-to-end application along with a steep learning curve associated with learning the nuances of each underlying technology. In this article, we present the Dynamic Distributed Dimensional Data Model (D4M) as a potential tool to unify database and storage engine operations. Previous articles on D4M have showcased the ability of D4M to interact with the popular NoSQL Accumulo database. Recently however, D4M now operates on a variety of backend storage or database engines while providing a federated look to the end user through the use of associative arrays. In order to showcase how new databases may be supported by D4M, we describe the process of building the D4M-SciDB connector and present performance of this connection. |
Year | DOI | Venue |
|---|---|---|
2015 | 10.1109/HPEC.2015.7322472 | 2015 IEEE High Performance Extreme Computing Conference (HPEC) |
Keywords | Field | DocType |
Big Data,Data Analytics,Dimensional Analysis,Federated Databases | Data mining,World Wide Web,Data administration,Database model,Computer science,View,Database design,NoSQL,Database engine,Database theory,Big data,Database | Journal |
Volume | ISSN | Citations |
abs/1508.07371 | 2377-6943 | 17 |
PageRank | References | Authors |
0.89 | 13 | 14 |
Name | Order | Citations | PageRank |
|---|---|---|---|
| Vijay Gadepally | 1 | 449 | 50.53 |
| Jeremy Kepner | 2 | 606 | 61.58 |
| William Arcand | 3 | 175 | 17.77 |
| David Bestor | 4 | 181 | 19.08 |
| Bill Bergeron | 5 | 168 | 16.57 |
| Chansup Byun | 6 | 180 | 19.21 |
| Lauren Edwards | 7 | 37 | 2.62 |
| Matthew Hubbell | 8 | 192 | 20.93 |
| Peter Michaleas | 9 | 201 | 20.93 |
| Julie Mullen | 10 | 138 | 15.22 |
| Andrew Prout | 11 | 182 | 18.78 |
| Antonio Rosa | 12 | 170 | 17.67 |
| Charles Yee | 13 | 147 | 15.14 |
| Albert Reuther | 14 | 335 | 37.32 |