Title
A case for RDMA in clouds: turning supercomputer networking into commodity
Abstract
Modern cloud computing infrastructures are steadily pushing the performance of their network stacks. At the hardware-level, already some cloud providers have upgraded parts of their network to 10GbE. At the same time there is a continuous effort within the cloud community to improve the network performance inside the virtualization layers. The low-latency/high-throughput properties of those network interfaces are not only opening the cloud for HPC applications, they will also be well received by traditional large scale web applications or data processing frameworks. However, as commodity networks get faster the burden on the end hosts increases. Inefficient memory copying in socket-based networking takes up a significant fraction of the end-to-end latency and also creates serious CPU load on the host machine. Years ago, the supercomputing community has developed RDMA network stacks like Infiniband that offer both low end-to-end latency as well as a low CPU footprint. While adopting RDMA to the commodity cloud environment is difficult (costly, requires special hardware) we argue in this paper that most of the benefits of RDMA can in fact be provided in software. To demonstrate our findings we have implemented and evaluated a prototype of a software-based RDMA stack. Our results, when compared to a socket/TCP approach (with TCP receive copy offload) show significant reduction in end-to-end latencies for messages greater than modest 64kB and reduction of CPU load (w/o TCP receive copy offload) for better efficiency while saturating the 10Gbit/s link.
Year
DOI
Venue
2011
10.1145/2103799.2103820
ApSys
Keywords
Field
DocType
cloud provider,rdma network stack,commodity network,modern cloud computing infrastructure,commodity cloud environment,cloud community,network performance,end-to-end latency,supercomputer network,network interface,copy offload,low latency,data processing,high throughput,cloud computing,rdma
Virtualization,InfiniBand,Supercomputer,Computer science,Remote direct memory access,S-LINK,Operating system,Network interface,Cloud computing,Network performance
Conference
Citations 
PageRank 
References 
11
0.81
16
Authors
3
Name
Order
Citations
PageRank
Animesh Trivedi1745.69
Bernard Metzler2897.72
Patrick Stuedi326118.76