Paper Info
Title
StorkCloud: data transfer scheduling and optimization as a service.
Abstract
Wide-area transfer of large data sets is still a big challenge despite the deployment of high-bandwidth networks with speeds reaching 100 Gbps. Most users fail to obtain even a fraction of theoretical speeds promised by these networks. Effective usage of the available network capacity has become increasingly important for wide-area data movement. We have developed a "data transfer scheduling and optimization system as a Cloud-hosted service", StorkCloud, which will mitigate the large-scale end-to-end data movement bottleneck by efficiently utilizing underlying networks and effectively scheduling and optimizing data transfers. In this paper, we present the initial design and prototype implementation of StorkCloud, and show its effectiveness in large dataset transfers across geographically distant storage sites, data centers, and collaborating institutions.
Year
DOI
Venue
2013
10.1145/2465848.2465855
ScienceCloud@HPDC
Keywords
Field
DocType
available network capacity,data transfer scheduling,cloud-hosted service,optimizing data transfer,large-scale end-to-end data movement,data center,large data set,wide-area transfer,wide-area data movement,large dataset transfer,big data,cloud computing
Bottleneck,Data set,Software deployment,Fair-share scheduling,Data transmission,Scheduling (computing),Computer science,Real-time computing,Big data,Cloud computing,Distributed computing
Conference
Citations 
PageRank 
References 
19
0.81
21
Authors
4
Name
Order
Citations
PageRank
Kosar, Tevfik161448.67
Engin Arslan211612.12
Brandon Ross3492.90
Bing Zhang4241.92