Paper Info
Title
Speculative Container Scheduling for Deep Learning Applications in a Kubernetes Cluster
Abstract
In the past decade, we have witnessed a dramatically increasing volume of data collected from various sources. To maximize utilization, various machine and deep learning models have been developed to study data. While data-driven applications improve countless products, hyperparameter tuning for the models is still a time-consuming and resource-intensive process. Cloud computing provides infrastructure support for the training of deep learning applications. The cloud service providers create an isolated virtual environment for clients who share physical resources, e.g., CPU and memory. On the cloud, resource management schemes are implemented to enable better sharing among users and boost system-wide performance. However, general scheduling approaches, such as spread priority and balanced resource schedulers, do not work well with deep learning workloads. In this article, we propose SpeCon, a novel container scheduler optimized for short-lived deep learning applications. Based on virtualized containers, such as Kubernetes and Docker, SpeCon analyzes the typical characteristics of training processes. We design a suite of algorithms to monitor the training’s progress and speculatively migrate the slow-growing models to release resources for fast-growing ones. Specifically, the extensive experiments demonstrate that SpeCon improves an individual job’s completion time by up to 41.5%, 14.8% system-wide, and 24.7% in terms of makespan.
Year
DOI
Venue
2022
10.1109/JSYST.2021.3129974
IEEE Systems Journal
Keywords
DocType
Volume
Apache Yarn and Spark,container management,Docker Swarm,Kubernetes,Pytorch,Tensorflow
Journal
16
Issue
ISSN
Citations 
3
1932-8184
0
PageRank 
References 
Authors
0.34
11
6
Name
Order
Citations
PageRank
Ying Mao1308.54
Yuqi Fu200.34
Wenjia Zheng300.34
Long Cheng49116.99
Qingzhi Liu510.70
Dingwen Tao612917.66