Name
Abstract | ||
|---|---|---|
This paper explores novel techniques for improving the performance of many-task workflows based on the Swift scripting language. We propose novel programmer options for automated distributed data placement and task scheduling. These options trigger a data placement mechanism used for distributing intermediate workflow data over the servers of Hercules, a distributed key-value store that can be used to cache file system data. We demonstrate that these new mechanisms can significantly improve the aggregated throughput of many-task workflows with up to 86x, reduce the contention on the shared file system, exploit the data locality, and trade off locality and load balance. |
Year | DOI | Venue |
|---|---|---|
2016 | 10.1109/CCGrid.2016.40 | 2016 16th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing (CCGrid) |
Keywords | Field | DocType |
scientific workflows,file systems,data locality,load balance,high performance | Locality,File system,Scheduling (computing),Load balancing (computing),Cache,Computer science,Server,Workflow,Operating system,Distributed computing,Scripting language | Conference |
ISSN | ISBN | Citations |
2376-4414 | 978-1-5090-2454-4 | 3 |
PageRank | References | Authors |
0.42 | 7 | 6 |
Name | Order | Citations | PageRank |
|---|---|---|---|
| Francisco Rodrigo Duro | 1 | 28 | 3.34 |
| Javier García | 2 | 47 | 9.85 |
| Florin Isaila | 3 | 234 | 24.01 |
| Justin M. Wozniak | 4 | 464 | 35.32 |
| Jesús Carretero | 5 | 552 | 69.87 |
| Robert Ross | 6 | 2717 | 173.13 |