Title
A unified storage layer for supporting distributed workflows in kubernetes
Abstract
ABSTRACTThe distributed computing landscape has been undergoing radical changes: High-Performance Computing (HPC) applications are moving to the Cloud, as a way of simplifying development, deployment, and migration across computing systems. Meanwhile, cloud applications are becoming increasingly complex and computationally intensive, with the advent of High-Performance Data Analytics (HPDA) pipelines --- highly distributed workflows dealing with enormous and diverse datasets, heavily relying on virtualization and containers, that would benefit from technologies used in "traditional" HPC. The foreseeable convergence, demands new abstractions to cope with the increased heterogeneity, so that differing workload classes can coexist seamlessly on the same infrastructure. In this paper, we propose a Unified Storage Layer (USL), to enable cloud-native applications to transparently access a wide spectrum of storage solutions, ranging from high-performance filesystems to cloud-based object stores and key-value databases. Allowing software to exploit the best out of the storage services available with no need for manual intervention from users or programmers, simplifies development and execution of workflows, and boosts overall productivity. This work has been motivated by the requirements of real-world, industry-driven applications running on Kubernetes, the industry standard for cloud infrastructure orchestration.
Year
DOI
Venue
2021
10.1145/3439839.3458735
EUROSYS
DocType
Citations 
PageRank 
Conference
0
0.34
References 
Authors
0
5
Name
Order
Citations
PageRank
Antony Chazapis100.68
Christian Pinto202.03
Yiannis Gkoufas300.34
Christos Kozyrakis45817355.99
Angelos Bilas500.68