Title
A job self-scheduling policy for HPC infrastructures
Abstract
The number of distributed high performance computing architectures has increased exponentially these last years. Thus, systems composed by several computational resources provided by different Research centers and Universities have become very popular. Job scheduling policies have been adapted to these new scenarios in which several independent resources have to be managed. New policies have been designed to take into account issues like multi-cluster environments, heterogeneous systems and the geographical distribution of the resources. Several centralized scheduling solutions have been proposed in the literature for these environments, such as centralized schedulers, centralized queues and global controllers. These approaches use a unique scheduling entity responsible for scheduling all the jobs that are submitted by the users. In this paper we propose the usage of self-scheduling techniques for dispatching the jobs that are submitted to a set of distributed computational hosts that are managed by independent schedulers (such as MOAB or LoadLeveler). It is a non-centralized and job-guided scheduling policy whose main goal is to optimize the job wait time. Thus, the scheduling decisions are done independently for each job instead of using a global policy where all the jobs are considered. On top of this, as a part of the proposed solution, we also demonstrate how the usage of job wait time prediction techniques can substantially improve the performance obtained in the described architecture.
Year
DOI
Venue
2007
10.1007/978-3-540-78699-3_4
JSSPP
Keywords
Field
DocType
hpc infrastructure,unique scheduling entity,centralized schedulers,centralized queue,job wait time prediction,job scheduling policy,job wait time,scheduling decision,centralized scheduling solution,computational host,job-guided scheduling policy,distributed computing,iron,job scheduling
Lottery scheduling,Fair-share scheduling,Computer science,Flow shop scheduling,Real-time computing,Two-level scheduling,Least slack time scheduling,Rate-monotonic scheduling,Dynamic priority scheduling,Round-robin scheduling,Distributed computing
Conference
Volume
ISSN
ISBN
4942
0302-9743
3-540-78698-8
Citations 
PageRank 
References 
15
0.74
34
Authors
2
Name
Order
Citations
PageRank
Francesc Guim11059.24
Julita Corbalan217412.00