Title
Optimizing Large Data Transfers Over 100gbps Wide Area Networks
Abstract
The Advanced Networking Initiative (ANI) project from the Energy Services Network provides a 100 Gbps testbed, which offers the opportunity for evaluating applications and middleware used by scientific experiments. This testbed is a prototype of a 100 Gbps wide-area network backbone, which links several Department of Energy (DOE) national laboratories, universities and other research institutions. These scientific experiments involve movement of large datasets for collaborations among researchers at different sites and thus require advanced infrastructure for supporting large and fast data transfers. A 100 Gbps network testbed is a key component of the ANI project and is used for DOE's science research programs. This work presents results towards obtaining maximum throughput in large data transfers by optimizing and fine-tuning scientific applications and middleware to use this advanced infrastructure efficiently. A detailed performance evaluation is discussed measuring both applications, from High Energy Physics (HEP) and from data transfer middleware (GridFTP, Globus Online, Storage Resource Management, XrootD and Squid) at 100 Gbps speeds and 53 ms of latency. Results show that up to 97% efficiency of such high bandwidth high latency network is possible, achieving 80-90 Gbps in most test cases with a peak transfer rate of 100 Gbps.
Year
DOI
Venue
2013
10.1109/CCGrid.2013.25
PROCEEDINGS OF THE 2013 13TH IEEE/ACM INTERNATIONAL SYMPOSIUM ON CLUSTER, CLOUD AND GRID COMPUTING (CCGRID 2013)
Keywords
Field
DocType
ANI Testbed, Grid Computing, Big Data, Caching, Wide Area Network
Middleware,Computer architecture,Grid computing,Computer science,Test case,GridFTP,Throughput,Wide area network,Backbone network,Big data,Operating system
Conference
ISSN
Citations 
PageRank 
2376-4414
1
0.38
References 
Authors
5
6
Name
Order
Citations
PageRank
Anupam Rajendran1752.48
Parag Mhashilkar2272.49
Hyunwoo Kim310.38
David Dykstra451.12
Gabriele Garzoglio5566.60
Raicu, Ioan62264129.28