Name
Title | ||
|---|---|---|
Performance Trade-offs in GPU Communication: A Study of Host and Device-initiated Approaches |
Abstract | ||
|---|---|---|
Network communication on GPU-based systems is a significant roadblock for many applications with small but frequent messaging requirements. One common question for application developers is, "How can they reduce the overheads and achieve the best communication performance on GPUs?" This work examines device initiated versus host initiated inter-node GPU communication using NVSHMEM. We derive basic communication model parameters for single message and batched communication before validating our model against distributed GEMM benchmarks. We use our model to estimate performance benefits for applications transitioning from CPUs to GPUS for fixed-size and scaled workloads and provide general guidelines for reducing communication overheads. Our findings show that the host-initiated approach generally outperforms the device-initiated approach for the system evaluated. |
Year | DOI | Venue |
|---|---|---|
2020 | 10.1109/PMBS51919.2020.00016 | 2020 IEEE/ACM Performance Modeling, Benchmarking and Simulation of High Performance Computer Systems (PMBS) |
Keywords | DocType | ISBN |
Performance evaluation,High Performance Computing,GPU Communication,SHMEM | Conference | 978-1-6654-2266-6 |
Citations | PageRank | References |
0 | 0.34 | 0 |
Authors | ||
8 | ||
Name | Order | Citations | PageRank |
|---|---|---|---|
| Taylor L. Groves | 1 | 26 | 7.20 |
| Ben Brock | 2 | 0 | 0.34 |
| Yuxin Chen | 3 | 1 | 0.81 |
| Khaled Z. Ibrahim | 4 | 215 | 21.25 |
| Lenny Oliker | 5 | 0 | 0.34 |
| Nicholas J. Wright | 6 | 408 | 27.79 |
| Samuel Williams | 7 | 1282 | 98.56 |
| Katherine A. Yelick | 8 | 3494 | 407.23 |