Paper Info
Title
Device-Driven Metadata Management Solutions for Scientific Big Data Use Cases
Abstract
Big Data applications in science are producing huge amounts of data, which require advanced processing, handling, and analysis capabilities. For the organization of large scale data sets it is essential to annotate these with metadata, index them, and make them easily findable. In this paper we investigate two scientific use cases from biology and photon science, which entail complex situations in regard to data volume, data rates and analysis requirements. The LSDMA project provides an ideal context for this research, combining both innovative R&D on the processing, handling, and analysis level and a wide range of research communities in need of scalable solutions. To facilitate the advancement of data life cycles we present preferred metadata management strategies. In biology the Open Microscopy Environment (OME) and in photon science NeXus/ICAT are presented. We show that these are well suited for the respective data life cycles. To facilitate searching across communities we discuss solutions involving the Open Archive Initiative - Protocol for Metadata Harvesting (OAI-PMH) and Apache Lucene/Solr.
Year
DOI
Venue
2014
10.1109/PDP.2014.119
PDP
Keywords
Field
DocType
biology computing,data analysis,meta data,optical microscopy,research and development,Apache Lucene,Apache Solr,LSDMA project,NeXus-ICAT,OAI-PMH,OME,advanced processing capability,analysis capability,biology,data analysis requirements,data life cycles,data rates,data set organization,data volume,device-driven metadata management solutions,handling capability,innovative R&D,light microscopy,open archive initiative-protocol-for-metadata harvesting,open microscopy environment,photon science,scientific big data use cases,Light Microscopy,Metadata Management,Photon Science,Scientific Big Data
Data science,Metadata,Protocol for Metadata Harvesting,World Wide Web,Use case,Meta Data Services,Data mapping,Data element,Computer science,Metadata management,Big data
Conference
ISSN
Citations 
PageRank 
1066-6192
0
0.34
References 
Authors
0
12
Name
Order
Citations
PageRank
Richard Grunzke17213.05
J眉rgen Hesser200.34
J眉rgen Starek300.34
Nick Kepper482.95
Sandra Gesing512125.55
Marcus Hardt6185.51
Volker Hartmann7146.94
Stephan Kindermann8103.02
Jan Potthoff942.66
Michael Hausmann1041.34
Ralph M眉ller-Pfefferkorn1100.34
Ren茅 J盲kel1200.34