Paper Info
Title
Dynamic protocol tuning algorithms for high performance data transfers
Abstract
Obtaining optimal data transfer performance is of utmost importance to today's data-intensive distributed applications and wide-area data replication services. Doing so necessitates effectively utilizing available network bandwidth and resources, yet in practice transfers seldom reach the levels of utilization they potentially could. Tuning protocol parameters such as pipelining, parallelism, and concurrency can significantly increase utilization and performance, however determining the best settings for these parameters is a difficult problem, as network conditions can vary greatly between sites and over time. In this paper, we present four application-level algorithms for heuristically tuning protocol parameters for data transfers in wide-area networks. Our algorithms dynamically tune the number of parallel data streams per file, the level of control channel pipelining, and the number of concurrent file transfers to fill network pipes. The presented algorithms are implemented as a standalone service as well as being used in interaction with external data scheduling tools such as Stork. The experimental results are very promising, and our algorithms outperform existing solutions in this area.
Year
DOI
Venue
2013
10.1007/978-3-642-40047-6_72
international conference on parallel processing
Keywords
DocType
Volume
high performance data transfer,network condition,wide-area data,wide-area network,parallel data stream,optimal data transfer performance,algorithms dynamically,external data,network pipe,available network bandwidth,data transfer,dynamic protocol
Conference
8097
ISSN
Citations 
PageRank 
0302-9743
9
0.51
References 
Authors
14
3
Name
Order
Citations
PageRank
Engin Arslan111612.12
Brandon Ross2492.90
Kosar, Tevfik361448.67