Paper Info
Title
ISABELA for effective in situ compression of scientific data.
Abstract
Exploding dataset sizes from extreme-scale scientific simulations necessitates efficient data management and reduction schemes to mitigate I/O costs. With the discrepancy between I/O bandwidth and computational power, scientists are forced to capture data infrequently, thereby making data collection an inherently lossy process. Although data compression can be an effective solution, the random nature of real-valued scientific datasets renders lossless compression routines ineffective. These techniques also impose significant overhead during decompression, making them unsuitable for data analysis and visualization, which require repeated data access. To address this problem, we propose an effective method for In situ Sort-And-B-spline Error-bounded Lossy Abatement (ISABELA) of scientific data that is widely regarded as effectively incompressible. With ISABELA, we apply a pre-conditioner to seemingly random and noisy data along spatial resolution to achieve an accurate fitting model that guarantees a >= 0.99 correlation with the original data. We further take advantage of temporal patterns in scientific data to compress data by approximate to 85%, while introducing only a negligible overhead on simulations in terms of runtime. ISABELA significantly outperforms existing lossy compression methods, such as wavelet compression, in terms of data reduction and accuracy. We extend upon our previous paper by additionally building a communication-free, scalable parallel storage framework on top of ISABELA-compressed data that is ideally suited for extreme-scale analytical processing. The basis for our storage framework is an inherently local decompression method (it need not decode the entire data), which allows for random access decompression and low-overhead task division that can be exploited over heterogeneous architectures. Furthermore, analytical operations such as correlation and query processing run quickly and accurately over data in the compressed space. Copyright (C) 2012 John Wiley & Sons, Ltd.
Year
DOI
Venue
2013
10.1002/cpe.2887
CONCURRENCY AND COMPUTATION-PRACTICE & EXPERIENCE
Keywords
DocType
Volume
lossy compression,B-spline,in situ processing,data-intensive application,high performance computing
Journal
25
Issue
ISSN
Citations 
SP4
1532-0626
25
PageRank 
References 
Authors
0.98
10
9
Name
Order
Citations
PageRank
Sriram Lakshminarasimhan118710.01
Neil Shah232324.15
Stéphane Ethier3958.15
Seung-Hoe Ku4875.90
Choong-Seock Chang5726.56
Scott Klasky6154799.00
Robert Latham736526.39
Robert Ross82717173.13
Nagiza F. Samatova986174.04