Name
Abstract | ||
|---|---|---|
We present Mesos, a platform for sharing commodity clusters between multiple diverse cluster computing frameworks, such as Hadoop and MPI. Sharing improves cluster utilization and avoids per-framework data replication. Mesos shares resources in a fine-grained manner, allowing frameworks to achieve data locality by taking turns reading data stored on each machine. To support the sophisticated schedulers of today's frameworks, Mesos introduces a distributed two-level scheduling mechanism called resource offers. Mesos decides how many resources to offer each framework, while frameworks decide which resources to accept and which computations to run on them. Our results show that Mesos can achieve near-optimal data locality when sharing the cluster among diverse frameworks, can scale to 50,000 (emulated) nodes, and is resilient to failures. |
Year | Venue | Keywords |
|---|---|---|
2011 | NSDI | data locality,multiple diverse cluster computing,data center,mesos shares resource,fine-grained resource sharing,avoids per-framework data replication,present mesos,commodity cluster,fine-grained manner,diverse framework,cluster utilization,near-optimal data,resource sharing |
Field | DocType | Citations |
Cluster (physics),Locality,Replication (computing),Computer science,Scheduling (computing),Real-time computing,Shared resource,Data center,Computer cluster,Distributed computing | Conference | 522 |
PageRank | References | Authors |
22.98 | 33 | 8 |
Name | Order | Citations | PageRank |
|---|---|---|---|
| Benjamin Hindman | 1 | 965 | 46.25 |
| Andy Konwinski | 2 | 3400 | 158.39 |
| Matei Zaharia | 3 | 9101 | 407.89 |
| Ali Ghodsi | 4 | 3306 | 156.01 |
| D. Joseph | 5 | 5463 | 492.96 |
| Randy H. Katz | 6 | 16819 | 3018.89 |
| Scott Shenker | 7 | 29892 | 2677.04 |
| Scott Shenker | 8 | 29892 | 2677.04 |