Name
Title | ||
|---|---|---|
Topology-Aware Optimization of Communications for Parallel Matrix Multiplication on Hierarchical Heterogeneous HPC Platform |
Abstract | ||
|---|---|---|
Communications on hierarchical heterogeneous HPC platforms can be optimized based on topology information. For MPI, as a major programming tool for such platforms, a number of topology-aware implementations of collective operations have been proposed for optimal scheduling of messages. This approach improves communication performance and does not require to modify application source code. However, it is applicable to collective operations only and does not affect the parts of the application that are based on point-to-point exchanges. In this paper, we address the problem of efficient execution of data-parallel applications on interconnected clusters and present a topology-aware optimization that improves data partition by taking into account the entire communication flow of the application. This approach is also non-intrusive to the source code but application-specific. For illustration, we use parallel matrix multiplication, where the matrices are partitioned into irregular 2D rectangles assigned to different processors and arranged in columns, and the processors communicate over this partitioning vertically and horizontally. By rearranging the rectangles, we can minimize communications between different levels of the network hierarchy. Finding the optimal arrangement is NP-complete, therefore, we propose a heuristic based on evaluation of the communication flow on the given topology. We demonstrate the correctness and efficiency of the proposed approach by experimental results on multicore nodes and interconnected heterogeneous clusters. |
Year | DOI | Venue |
|---|---|---|
2014 | 10.1109/IPDPSW.2014.10 | Parallel & Distributed Processing Symposium Workshops |
Keywords | Field | DocType |
computational complexity,matrix multiplication,message passing,optimisation,parallel processing,processor scheduling,source code (software),topology,MPI,NP-complete,data-parallel applications,hierarchical heterogeneous HPC platform,optimal message scheduling,parallel matrix multiplication,source code,topology-aware communication optimization,data partitioning,heterogeneous clusters,matrix multiplication,topology-aware communications | Topology,Logical topology,Computer science,Source code,Parallel computing,Network topology,Hierarchical network model,Topology optimization,Multi-core processor,Matrix multiplication,Computational topology,Distributed computing | Conference |
Citations | PageRank | References |
0 | 0.34 | 14 |
Authors | ||
4 | ||
Name | Order | Citations | PageRank |
|---|---|---|---|
| Tania Malik | 1 | 0 | 0.34 |
| Vladimir Rychkov | 2 | 0 | 0.34 |
| Alexey L. Lastovetsky | 3 | 6 | 0.93 |
| Jean-Noël Quintin | 4 | 28 | 4.11 |