Title
Overlapping communications in gyrokinetic codes on accelerator‐based platforms
Abstract
Communication and computation overlapping techniques have been introduced in the five-dimensional gyrokinetic codes GYSELA and GKV. In order to anticipate some of the exa-scale requirements, these codes were ported to the modern accelerators, Xeon Phi KNL and Tesla P 100 GPU. On accelerators, a serial version of GYSELA on KNL and GKV on GPU are respectively 1.3x and 7.4x faster than those on a single Skylake processor (a single socket). For the scalability, we have measured GYSELA performance on Xeon Phi KNL from 16 to 512 KNLs (1024 to 32k cores) and GKV performance on Tesla P 100 GPU from 32 to 256 GPUs. In their parallel versions, transpose communication in semi-Lagrangian solver in GYSELA or Convolution kernel in GKV turned out to be a main bottleneck. This indicates that in the exa-scale, the network constraints would be critical. In order to mitigate the communication costs, the pipeline and task-based overlapping techniques have been implemented in these codes. The GYSELA 2D advection solver has achieved a 33% to 92% speed up, and the GKV 2D convolution kernel has achieved a factor of 2 speed up with pipelining. The task-based approach gives 11% to 82% performance gain in the derivative computation of the electrostatic potential in GYSELA. We have shown that the pipeline-based approach is applicable with the presence of symmetry, while the task-based approach can be applicable to more general situations.
Year
DOI
Venue
2020
10.1002/cpe.5551
CONCURRENCY AND COMPUTATION-PRACTICE & EXPERIENCE
Keywords
Field
DocType
overlap,semi-Lagrangian,spectral,Tesla P100 GPU,transpose communication,Xeon Phi KNL
Computer science,Distributed computing
Journal
Volume
Issue
ISSN
32.0
5.0
1532-0626
Citations 
PageRank 
References 
0
0.34
0
Authors
6
Name
Order
Citations
PageRank
Yuuichi Asahi101.35
Guillaume Latu26716.21
julien bigot3135.50
Shinya Maeyama400.34
V. Grandgirard56617.91
Yasuhiro Idomura65714.27