Paper Info
Title
Mission possible: Unify HPC and Big Data stacks towards application-defined blobs at the storage layer
Abstract
HPC and Big Data stacks are completely separated today. The storage layer offers opportunities for convergence, as the challenges associated with HPC and Big Data storage are similar: trading versatility for performance. This motivates a global move towards dropping file-based, POSIX-IO compliance systems. However, on HPC platforms this is made difficult by the centralized storage architecture using file-based storage. In this paper we advocate that the growing trend of equipping HPC compute nodes with local storage redistributes the cards by enabling object storage to be deployed alongside the application on the compute nodes. Such integration of application and storage not only allows fine-grained configuration of the storage system, but also improves application portability across platforms. In addition, the single-user nature of such application-specific storage obviates the need for resource-consuming storage features like permissions or file hierarchies offered by traditional file systems. In this article we propose and evaluate Blobs (Binary Large Objects) as an alternative to distributed file systems. We factually demonstrate that it offers drop-in compatibility with a variety of existing applications while improving storage throughput by up to 28%.
Year
DOI
Venue
2020
10.1016/j.future.2018.07.035
Future Generation Computer Systems
DocType
Volume
ISSN
Journal
109
0167-739X
Citations 
PageRank 
References 
0
0.34
24
Authors
9
Name
Order
Citations
PageRank
Pierre Matri182.14
Yevhen Alforov221.05
Alvaro Brandon361.85
María S Pérez4403.46
Alexandru Costan518223.43
Gabriel Antoniu680964.62
Michael Kuhn74211.42
Philip H. Carns896462.51
Thomas Ludwig, II94512.12