Paper Info
Title
Fast Olap Query Execution In Main Memory On Large Data In A Cluster
Abstract
Main memory column-stores have proven to be efficient for processing analytical queries. Still, there has been little work in the context of clusters. Using only a single machine poses several restrictions: Processing power and data volume are bounded to the number of cores and main memory fitting on one tightly coupled system. To enable the processing of larger data sets, switching to a cluster becomes necessary. In this work, we explore techniques for efficient execution of analytical SQL queries on large amounts of data in a parallel database cluster while making maximal use of the available hardware. This includes precompiled query plans for efficient CPU utilization, full parallelization on single nodes and across the cluster, and efficient inter-node communication. We implement all features in a prototype for running a subset of TPC-H benchmark queries. We evaluate our implementation in a 128 node cluster running TPC-H queries with 30 000 gigabyte of uncompressed data. Currently, there are no official cluster results for more than 10 000 gigabyte of data, where we achieve up to one to two orders of magnitudes better performance than the current record holder.
Year
Venue
Keywords
2013
2013 IEEE INTERNATIONAL CONFERENCE ON BIG DATA
Distributed databases, Distributed computing, Parallel processing, Query processing, Data analysis, Data warehouses
Field
DocType
ISSN
Query optimization,Data mining,Query language,Parallel database,Computer science,Sargable,Parallel computing,In-Memory Processing,Query by Example,Spatial query,Online analytical processing
Conference
2639-1589
Citations 
PageRank 
References 
6
0.51
14
Authors
3
Name
Order
Citations
PageRank
Martin Weidner191.01
Jonathan Dees21637.51
Peter Sanders31957120.14