Name
Abstract | ||
|---|---|---|
To enable the rapid execution of many tasks on compute clusters, we have developed Falkon, a Fast and Light-weight tasK executiON framework. Falkon integrates (1) multi-level scheduling to separate resource acquisition (via, e.g., requests to batch schedulers) from task dispatch, and (2) a streamlined dispatcher. Falkon's integration of multi-level scheduling and streamlined dispatchers delivers performance not provided by any other system. We describe Falkon architecture and implementation, and present performance results for both microbenchmarks and applications. Microbenchmarks show that Falkon throughput (487 tasks/sec) and scalability (to 54,000 executors and 2,000,000 tasks processed in just 112 minutes) are one to two orders of magnitude better than other systems used in production Grids. Large-scale astronomy and medical applications executed under Falkon by the Swift parallel programming system achieve up to 90% reduction in end-to-end run time, relative to versions that execute tasks via separate scheduler submissions. |
Year | DOI | Venue |
|---|---|---|
2007 | 10.1145/1362622.1362680 | SC |
Keywords | Field | DocType |
falkon throughput,streamlined dispatcher,separate resource acquisition,multi-level scheduling,present performance result,swift parallel programming system,batch schedulers,separate scheduler submission,light-weight task execution framework,falkon architecture,scalability,security,computer science,data mining,protocols,throughput,production systems,parallel programming,scheduling,computer architecture,government,grid computing | Many-task computing,Grid computing,Swift,Scheduling (computing),Computer science,Parallel computing,Throughput,Pilot job,Processor scheduling,Operating system,Scalability,Distributed computing | Conference |
Citations | PageRank | References |
171 | 9.42 | 31 |
Authors | ||
5 | ||
Name | Order | Citations | PageRank |
|---|---|---|---|
| Raicu, Ioan | 1 | 2264 | 129.28 |
| Yong Zhao | 2 | 171 | 9.42 |
| Catalin Dumitrescu | 3 | 737 | 56.44 |
| Foster Ian | 4 | 22938 | 2663.24 |
| Mike Wilde | 5 | 351 | 22.09 |