Name
Abstract | ||
|---|---|---|
We present a query processing framework for the efficient evaluation of spatial filters on large numerical simulation datasets stored in a data-intensive cluster. Previously, filtering of large numerical simulations stored in scientific databases has been impractical owing to the immense data requirements. Rather, filtering is done during simulation or by loading snapshots into the aggregate memory of an HPC cluster. Our system performs filtering within the database and supports large filter widths. We present two complementary methods of execution: I/O streaming computes a batch filter query in a single sequential pass using incremental evaluation of decomposable kernels, summed volumes generates an intermediate data set and evaluates each filtered value by accessing only eight points in this dataset. We dynamically choose between these methods depending upon workload characteristics. The system allows us to perform filters against large data sets with little overhead: query performance scales with the cluster's aggregate I/O throughput. |
Year | DOI | Venue |
|---|---|---|
2012 | 10.1109/SC.2012.41 | SC |
Keywords | Field | DocType |
parallel processing,hpc cluster,query processing framework,query performance scale,large numerical simulation,data-intensive spatial filtering,information filtering,numerical analysis,data-intensive spatial,large data set,intermediate data,decomposable kernels,single sequential pass,aggregate memory,large filter width,batch filter query,i/o streaming,data-intensive cluster,large numerical simulation datasets,immense data requirement,query processing,optimization,quantum chemistry,heuristic algorithm | Data set,Computer simulation,Computer science,Heuristic (computer science),Parallel computing,Filter (signal processing),Throughput,Numerical analysis,Snapshot (computer storage),Distributed computing,Spatial filter | Conference |
ISSN | ISBN | Citations |
2167-4329 | 978-1-4673-0805-2 | 3 |
PageRank | References | Authors |
0.44 | 15 | 5 |
Name | Order | Citations | PageRank |
|---|---|---|---|
| Kalin Kanov | 1 | 11 | 3.06 |
| Randal Burns | 2 | 1955 | 115.15 |
| Greg Eyink | 3 | 3 | 0.44 |
| Charles Meneveau | 4 | 61 | 5.72 |
| Alexander Szalay | 5 | 124 | 10.16 |