Paper Info
Title
A methodology for evaluating the impact of data compression on climate simulation data
Abstract
High-resolution climate simulations require tremendous computing resources and can generate massive datasets. At present, preserving the data from these simulations consumes vast storage resources at institutions such as the National Center for Atmospheric Research (NCAR). The historical data generation trends are economically unsustainable, and storage resources are already beginning to limit science objectives. To mitigate this problem, we investigate the use of data compression techniques on climate simulation data from the Community Earth System Model. Ultimately, to convince climate scientists to compress their simulation data, we must be able to demonstrate that the reconstructed data reveals the same mean climate as the original data, and this paper is a first step toward that goal. To that end, we develop an approach for verifying the climate data and use it to evaluate several compression algorithms. We find that the diversity of the climate data requires the individual treatment of variables, and, in doing so, the reconstructed data can fall within the natural variability of the system, while achieving compression rates of up to 5:1.
Year
DOI
Venue
2014
10.1145/2600212.2600217
HPDC
Keywords
Field
DocType
data compaction and compression,data compression,high performance computing,software/program verification
Data mining,Climate simulation,Supercomputer,Computer science,Community earth system model,Data compression,Test data generation
Conference
Citations 
PageRank 
References 
23
1.06
10
Authors
9
Name
Order
Citations
PageRank
Allison H. Baker1261.98
Haiying Xu2272.45
John M. Dennis324139.70
Michael N. Levy4291.79
Doug Nychka5231.06
Sheri A. Mickelson6354.32
Jim Edwards7774.61
Mariana Vertenstein8847.17
Al Wegener9231.06