Paper Info
Title
I/O-aware bandwidth allocation for petascale computing systems.
Abstract
In the Big Data era, the gap between the storage performance and an application's I/O requirement is increasing. I/O congestion caused by concurrent storage accesses from multiple applications is inevitable and severely harms the performance. Conventional approaches either focus on optimizing an application's access pattern individually or handle I/O requests on a low-level storage layer without any knowledge from the upper-level applications. In this paper, we present a novel I/O-aware bandwidth allocation framework to coordinate ongoing I/O requests on petascale computing systems. The motivation behind this innovation is that the resource management system has a holistic view of both the system state and jobs' activities and can dynamically control the jobs' status or allocate resource on the fly during their execution. We treat a job's I/O requests as periodical sub-jobs within its lifecycle and transform the I/O congestion issue into a classical scheduling problem. Based on this model, we propose a bandwidth management mechanism as an extension to the existing scheduling system. We design several bandwidth allocation policies with different optimization objectives either on user-oriented metrics or system performance. We conduct extensive trace-based simulations using real job traces and I/O traces from a production IBM Blue Gene/Q system at Argonne National Laboratory. Experimental results demonstrate that our new design can improve job performance by more than 30%, as well as increasing system performance.
Year
DOI
Venue
2016
10.1016/j.parco.2016.05.005
Parallel Computing
Keywords
Field
DocType
Job scheduling,Resource management,I/O congestion,Resource allocation
Job shop scheduling,Computer science,Bandwidth allocation,Parallel computing,Input/output,Resource Management System,Resource allocation,Job scheduler,Petascale computing,Bandwidth management
Journal
Volume
Issue
ISSN
58
C
0167-8191
Citations 
PageRank 
References 
2
0.36
22
Authors
7
Name
Order
Citations
PageRank
Zhou Zhou1856.02
Xu Yang2876.95
Dongfang Zhao336226.49
Paul Rich4508.21
Wei Tang515210.65
Jia Wang614812.47
Zhiling Lan781854.25