Title
Fine-Grained Processing Towards HL-LHC Computing in ATLAS
Abstract
During LHC's Run-2 ATLAS has been developing and evaluating new fine-grained approaches to workflows and dataflows able to better utilize computing resources in terms of storage, processing and networks. The compute-limited physics of ATLAS has driven the collaboration to aggressively harvest opportunistic cycles from what are often transiently available resources, including HPCs, clouds, volunteer computing, and grid resources in transitional states. Fine-grained processing (with typically a few minutes' granularity, corresponding to one event for the present ATLAS full simulation) enables agile workflows with a light footprint on the resource such that cycles can be more fully and efficiently utilized than with conventional workflows processing O(GB) files per job.
Year
DOI
Venue
2018
10.1109/eScience.2018.00083
2018 IEEE 14th International Conference on e-Science (e-Science)
Keywords
Field
DocType
workflow,dataflow,streaming,fine-grained,processing
Large Hadron Collider,Grid computing,Computer science,Agile software development,Dataflow,Granularity,Distributed database,Workflow,Payload,Distributed computing
Conference
ISSN
ISBN
Citations 
2325-372X
978-1-5386-9157-1
0
PageRank 
References 
Authors
0.34
0
16