Paper Info
Title
Qserv: a distributed shared-nothing database for the LSST catalog
Abstract
The LSST project will provide public access to a database catalog that, in its final year, is estimated to include 26 billion stars and galaxies in dozens of trillion detections in multiple petabytes. Because we are not aware of an existing open-source database implementation that has been demonstrated to efficiently satisfy astronomers' spatial self-joining and cross-matching queries at this scale, we have implemented Qserv, a distributed shared-nothing SQL database query system. To speed development, Qserv relies on two successful open-source software packages: the MySQL RDBMS and the Xrootd distributed file system. We describe Qserv's design, architecture, and ability to scale to LSST's data requirements. We illustrate its potential with test results on a 150-node cluster using 55 billion rows and 30 terabytes of simulated data. These results demonstrate the soundness of Qserv's approach and the scale it achieves on today's hardware.
Year
DOI
Venue
2011
10.1145/2063348.2063364
High Performance Computing, Networking, Storage and Analysis
Keywords
Field
DocType
file system,data requirement,lsst catalog,billion row,existing open-source database implementation,shared-nothing database,sql database query system,simulated data,billion star,lsst project,successful open-source software package,database catalog,distributed file system,satisfiability,public domain software,sql,database,hardware,indexing,bandwidth,servers,parallel,relational databases,shared nothing,distributed databases
SQL,Distributed File System,File system,Relational database,Computer science,Shared nothing architecture,Relational database management system,Distributed database,Database,Operating system,Database catalog
Conference
ISBN
Citations 
PageRank 
978-1-4503-0771-0
1
0.44
References 
Authors
5
4
Name
Order
Citations
PageRank
Daniel L. Wang13911.57
Serge M. Monkewitz210.44
Kian-Tat Lim311811.63
Jacek Becla417221.55