Name
Abstract | ||
|---|---|---|
In the Big Data era, the gap between the storage performance and an application's I/O requirement is increasing. I/O congestion caused by concurrent storage accesses from multiple applications is inevitable and severely harms the performance. Conventional approaches either focus on optimizing an application's access pattern individually or handle I/O requests on a low-level storage layer without any knowledge from the upper-level applications. In this paper, we present a novel I/O-aware batch scheduling framework to coordinate ongoing I/O requests on petascale computing systems. The motivation behind this innovation is that the batch scheduler has a holistic view of both the system state and jobs' activities and can control the jobs' status on the fly during their execution. We treat a job's I/O requests as periodical subjobs within its lifecycle and transform the I/O congestion issue into a classical scheduling problem. We design two scheduling polices with different scheduling objectives either on user-oriented metrics or system performance. We conduct extensive trace-based simulations using real job traces and I/O traces from a production IBM Blue Gene/Q system. Experimental results demonstrate that our design can improve job performance by more than 30%, as well as increasing system performance. |
Year | DOI | Venue |
|---|---|---|
2015 | 10.1109/CLUSTER.2015.45 | Cluster Computing |
Keywords | Field | DocType |
I/O-aware batch scheduling,petascale computing systems,low-level storage layer,input-output request handling,I/O congestion issue,scheduling policy,production IBM Blue Gene/Q system,trace-based simulation | I/O scheduling,Scheduling (computing),Computer science,Input/output,Two-level scheduling,Real-time computing,Rate-monotonic scheduling,Distributed computing,Fair-share scheduling,Parallel computing,Job scheduler,Dynamic priority scheduling,Operating system | Conference |
ISSN | Citations | PageRank |
1552-5244 | 9 | 0.57 |
References | Authors | |
32 | 7 | |
Name | Order | Citations | PageRank |
|---|---|---|---|
| Zhou Zhou | 1 | 85 | 6.02 |
| Xu Yang | 2 | 87 | 6.95 |
| Dongfang Zhao | 3 | 362 | 26.49 |
| Paul Rich | 4 | 50 | 8.21 |
| Wei Tang | 5 | 152 | 10.65 |
| Jia Wang | 6 | 148 | 12.47 |
| Zhiling Lan | 7 | 818 | 54.25 |