Name
Abstract | ||
|---|---|---|
YARN is a prevailing central resource management architecture, which allocates a group of resources to each application. The resource group is consistent with locality requests of tasks in applications. But each application allocates the resources in the resource group to each task according to locality capacities of resources. Different rules in two levels of resource allocations lead to mismatched localities for tasks, which hurts performances of applications. There is a lack of researches about mismatched localities for tasks in YARN. This paper designs a Proportion scheduler to improve mismatched localities without dragging applications. The locality capacities of resources becomes unified allocation rule. Resources are classified by the locality requests of tasks, so that resources in the same category are versatile for tasks with the same level of locality request. This classification decreases the mismatched probability. In addition, the improvement of mismatched localities makes compromises between different applications. Every application is assigned with proportional resources in different locality scales for improved performances. Compared to baseline schedulers, there are 2 times data-local tasks and more than 30% rack-local tasks. The Proportion decreases makespan of applications by a maximum 66.7% and network traffic by an utmost 80%. |
Year | DOI | Venue |
|---|---|---|
2018 | 10.1109/BDCloud.2018.00068 | 2018 IEEE Intl Conf on Parallel & Distributed Processing with Applications, Ubiquitous Computing & Communications, Big Data & Cloud Computing, Social Computing & Networking, Sustainable Computing & Communications (ISPA/IUCC/BDCloud/SocialCom/SustainCom) |
Keywords | Field | DocType |
YARN,mismatched locality,proportion | Resource management,Locality,Architecture,Yarn,Job shop scheduling,Computer science,Resource allocation,Multimedia,Distributed computing | Conference |
ISSN | ISBN | Citations |
2158-9178 | 978-1-7281-1141-4 | 0 |
PageRank | References | Authors |
0.34 | 0 | 4 |
Name | Order | Citations | PageRank |
|---|---|---|---|
| Jie Xu | 1 | 2 | 1.05 |
| J. Wang | 2 | 479 | 95.23 |
| Qi Qi | 3 | 210 | 56.01 |
| Jianxin Liao | 4 | 457 | 82.08 |