Paper Info
Title
The development of QC standards for ARM data products
Abstract
This paper describes a standard for the representation of data quality information relevant to a geophysical data set, and the specific implementation of this standard for the U.S. Department of Energy's Atmospheric Radiation Measurement (ARM) program. The standard is applicable to near real-time instrument data as well as episodic or batch-processed model data. It was designed especially to facilitate use of data quality checks or "QC" by automated processes but also simplifies interpretation and visualization of this data quality information for humans. The standard is extensible and customizable, permitting an arbitrary number of data quality checks, either global in nature or specific to individual data variables. For the ARM data set, this standard has been implemented in netCDF, which permits arbitrarily verbose descriptions of the data quality checks, while storing the results in a compact, efficient format. In addition, we describe tools developed for the creation and population of the QC fields, rule-based validation of adherence to the QC standards, and the display of QC information in ARM data sets.
Year
DOI
Venue
2010
10.1145/1851476.1851556
HPDC
Keywords
Field
DocType
near real-time instrument data,batch-processed model data,geophysical data,arm data,data quality check,arm data set,individual data variable,qc field,arm data product,data quality information,qc standard,rule based,atmospheric chemistry,data,data quality assurance,near real time,arm,batch process,data quality,quality control,environmental sciences
Instrument Data,Data mining,Population,Data set,Data quality,Computer science,Visualization,Atmospheric radiation,Data products,NetCDF,Database
Conference
Citations 
PageRank 
References 
2
0.44
2
Authors
4
Name
Order
Citations
PageRank
Krista Gaustad140.98
Connor Flynn220.78
Sherman Beus381.70
Brian Ermold440.98