Paper Info
Title
Understanding the Performance of a Prototype of a WLCG Data Lake for HL-LHC
Abstract
Storage is identified as one of the main challenges for WLCG in the next decade in the computing strategy document for HL-LHC [1]. Extrapolating todays computing models, the ATLAS and CMS experiments alone would need one order of magnitude more storage resources than what could be provided by the funding agencies. Organization and consolidation of storage and evolution of the compute facilities will be central in addressing the possible resources shortage. In this contribution we describe the architecture of a prototype of a WLCG data lake for HL-LHC. A WLCG data lake aims to provide a geographically distributed storage service, distributed across large data centres interconnected by fast networks with low latency. We present methodology used to measure and understand the performance of a WLCG data lake prototype, in order to compare event throughput at the same cost at the same compute facilities backed by the traditional storage services and backed by the WLCG data lake. We will discuss various possible data processing models w.r.t. network latency, available storage media, and data caching approaches. We will present benchmarks for storage service and compute performance.
Year
DOI
Venue
2018
10.1109/eScience.2018.00080
2018 IEEE 14th International Conference on e-Science (e-Science)
Keywords
Field
DocType
HL-LHC,WLCG,storage,performance,distributed computing
Large Hadron Collider,Data processing,Computer science,Distributed data store,Throughput,Latency (engineering),Distributed database,Economic shortage,Benchmark (computing),Distributed computing
Conference
ISSN
ISBN
Citations 
2325-372X
978-1-5386-9157-1
0
PageRank 
References 
Authors
0.34
0
6
Name
Order
Citations
PageRank
Jaroslava Schovancova100.34
Simone Campana2182.76
Xavier Espinal Curull300.34
Maria Girone410.77
Ivan Kadochnikov500.34
Gavin John McCance600.34