Name
Title | ||
|---|---|---|
Distributed Proximal Gradient Algorithm for Partially Asynchronous Computer Clusters. |
Abstract | ||
|---|---|---|
With ever growing data volume and model size, an error-tolerant, communication efficient, yet versatile distributed algorithm has become vital for the success of many large-scale machine learning applications. In this work we propose m-PAPG, an implementation of the flexible proximal gradient algorithm in model parallel systems equipped with the partially asynchronous communication protocol. The worker machines communicate asynchronously with a controlled staleness bound s and operate at different frequencies. We characterize various convergence properties of m-PAPG: 1) Under a general non-smooth and non-convex setting, we prove that every limit point of the sequence generated by m-PAPG is a critical point of the objective function; 2) Under an error bound condition of convex objective functions, we prove that the optimality gap decays linearly for every s steps; 3) Under the Kurdyka-Lojasiewicz inequality and a sufficient decrease assumption , we prove that the sequences generated by m-PAPG converge to the same critical point, provided that a proximal Lipschitz condition is satisfied. |
Year | Venue | Keywords |
|---|---|---|
2018 | JOURNAL OF MACHINE LEARNING RESEARCH | proximal gradient,distributed system,model parallel,partially asynchronous,machine learning |
Field | DocType | Volume |
Convergence (routing),Asynchronous communication,Algorithm,Regular polygon,Critical point (thermodynamics),Distributed algorithm,Lipschitz continuity,Limit point,Computer cluster,Mathematics | Journal | 19 |
ISSN | Citations | PageRank |
1532-4435 | 0 | 0.34 |
References | Authors | |
20 | 5 | |
Name | Order | Citations | PageRank |
|---|---|---|---|
| Yi Zhou | 1 | 65 | 17.55 |
| Yingbin Liang | 2 | 1646 | 147.64 |
| Yaoliang Yu | 3 | 669 | 34.33 |
| Wei Dai | 4 | 333 | 12.77 |
| Bo Xing | 5 | 7332 | 471.43 |