Name
Abstract | ||
|---|---|---|
Cluster-level thermal management has gained much attention over the past decade due to rising cooling costs associated with data centers. In this research, we propose and implement a static scheduler called SSched and a dynamic one named DSched. These 2 algorithms schedule jobs based on CPU and disk temperatures of a Hadoop cluster's nodes. Our schedulers rely on a monitoring mechanism to keep track of CPU and disk utilization, maintaining CPU and disk temperatures below a threshold through thermal-aware scheduling decisions. To facilitate the design of SSched and DSched, we classify jobs into the CPU-intensive and disk-intensive categories. When a job arrives, SSched retrieves the utilization stats from a profiled log, estimates the thermal behavior, and places the job on NodeManager to minimize thermal impacts. Unlike SSched, DSched improves thermal efficiency of Hadoop clusters through dynamic load balancing. DSched keeps track of the coolest and hottest nodes in the cluster; tasks are migrated from hot nodes into cool ones if any hot spot is detected. To evaluate the effectiveness of our schedulers, we keep track of average CPU and disk temperatures in a node, managing an optimal outlet temperature across a cluster. We demonstrate that compared with the traditional Hadoop scheduler, SSched and DSched achieve approximately 15% savings in terms of cooling cost with little performance overhead. |
Year | DOI | Venue |
|---|---|---|
2017 | 10.1002/cpe.4206 | CONCURRENCY AND COMPUTATION-PRACTICE & EXPERIENCE |
Keywords | Field | DocType |
benchmarking,CPU-intensive,data centers,disk-intensive job,Hadoop,MapReduce,task assignment,thermal-aware job scheduler,thermal profiling | Cluster (physics),Hot spot (veterinary medicine),Thermal,Thermal profiling,Thermal efficiency,Scheduling (computing),Computer science,Parallel computing,Real-time computing,Benchmarking,Distributed computing,High performance computing clusters | Journal |
Volume | Issue | ISSN |
29 | 18 | 1532-0626 |
Citations | PageRank | References |
0 | 0.34 | 7 |
Authors | ||
4 | ||
Name | Order | Citations | PageRank |
|---|---|---|---|
| Shubbhi Taneja | 1 | 2 | 2.72 |
| Sanjay Kulkarni | 2 | 4 | 1.12 |
| Yi Zhou | 3 | 230 | 32.97 |
| Xiao Qin | 4 | 1836 | 125.69 |