Paper Info
Title
A scalable approach to solving dense linear algebra problems on hybrid CPU‐GPU systems
Abstract
Aiming to fully exploit the computing power of all CPUs and all graphics processing units (GPUs) on hybrid CPU-GPU systems to solve dense linear algebra problems, we design a class of heterogeneous tile algorithms to maximize the degree of parallelism, to minimize the communication volume, and to accommodate the heterogeneity between CPUs and GPUs. The new heterogeneous tile algorithms are executed upon our decentralized dynamic scheduling runtime system, which schedules a task graph dynamically and transfers data between compute nodes automatically. The runtime system uses a new distributed task assignment protocol to solve data dependencies between tasks without any coordination between processing units. By overlapping computation and communication through dynamic scheduling, we are able to attain scalable performance for the double-precision Cholesky factorization and QR factorization. Our approach demonstrates a performance comparable to Intel MKL on shared-memory multicore systems and better performance than both vendor (e.g., Intel MKL) and open source libraries (e.g., StarPU) in the following three environments: heterogeneous clusters with GPUs, conventional clusters without GPUs, and shared-memory systems with multiple GPUs. Copyright (C) 2014 John Wiley & Sons, Ltd.
Year
DOI
Venue
2015
10.1002/cpe.3403
CONCURRENCY AND COMPUTATION-PRACTICE & EXPERIENCE
Keywords
Field
DocType
dense linear algebra,heterogeneous HPC systems,distributed dataflow scheduling,runtime systems
Computer science,CUDA,Degree of parallelism,Parallel computing,Schedule,Dynamic priority scheduling,QR decomposition,Cholesky decomposition,Runtime system,Distributed computing,Scalability
Journal
Volume
Issue
ISSN
27
14
1532-0626
Citations 
PageRank 
References 
5
0.58
21
Authors
2
Name
Order
Citations
PageRank
Fengguang Song123219.88
Jack J. Dongarra2176252615.79