Name
Title | ||
|---|---|---|
LOSP: Overlap Synchronization Parallel With Local Compensation for Fast Distributed Training |
Abstract | ||
|---|---|---|
When running in Parameter Server (PS), the Distributed Stochastic Gradient Descent (D-SGD) incurs significant communication delays and huge communication overhead due to the model synchronization. Moreover, considering the heterogeneity of computational capability among workers, traditional synchronization modes incur under-utilization of computational resources because fast workers have to wait f... |
Year | DOI | Venue |
|---|---|---|
2021 | 10.1109/JSAC.2021.3087272 | IEEE Journal on Selected Areas in Communications |
Keywords | DocType | Volume |
Synchronization,Computational modeling,Servers,Training,Delays,Convergence,Machine learning | Journal | 39 |
Issue | ISSN | Citations |
8 | 0733-8716 | 1 |
PageRank | References | Authors |
0.38 | 0 | 6 |
Name | Order | Citations | PageRank |
|---|---|---|---|
| Haozhao Wang | 1 | 4 | 1.77 |
| Zhihao Qu | 2 | 42 | 5.45 |
| Song Guo | 3 | 3431 | 278.71 |
| Ningqi Wang | 4 | 1 | 0.38 |
| Ruixuan Li | 5 | 405 | 69.47 |
| Zhuang, W. | 6 | 4190 | 302.05 |