Name
Abstract | ||
|---|---|---|
This paper proposes a framework of L-BFGS based on the (approximate) second-order information with stochastic batches, as a novel approach to the finite-sum minimization problems. Different from the classical L-BFGS where stochastic batches lead to instability, we use a smooth estimate for the evaluations of the gradient differences while achieving acceleration by well-scaling the initial Hessians. We provide theoretical analyses for both convex and nonconvex cases. In addition, we demonstrate that within the popular applications of least-square and cross-entropy losses, the algorithm admits a simple implementation in the distributed environment. Numerical experiments support the efficiency of our algorithms. |
Year | Venue | Field |
|---|---|---|
2018 | arXiv: Learning | Mathematical optimization,Distributed Computing Environment,Instability,Regular polygon,Minification,Acceleration,Broyden–Fletcher–Goldfarb–Shanno algorithm,Mathematics |
DocType | Volume | Citations |
Journal | abs/1807.05328 | 1 |
PageRank | References | Authors |
0.35 | 12 | 4 |
Name | Order | Citations | PageRank |
|---|---|---|---|
| Jie Liu | 1 | 61 | 3.25 |
| Yu Rong | 2 | 116 | 17.89 |
| Martin Takác | 3 | 752 | 49.49 |
| Junzhou Huang | 4 | 2182 | 141.43 |