Name
Abstract | ||
|---|---|---|
This work presents a new optimization algorithm for empirical risk minimization. The algorithm bridges the gap between first- and second-order methods by computing a search direction that uses a second-order-type update in one subspace, coupled with a scaled steepest descent step in the orthogonal complement. To this end, partial curvature information is incorporated to help with ill-conditioning, while simultaneously allowing the algorithm to scale to the large problem dimensions often encountered in machine learning applications. Theoretical results are presented to confirm that the algorithm converges to a stationary point in both the strongly convex and non-convex cases. A stochastic variant of the algorithm is also presented, along with corresponding theoretical guarantees. Numerical results confirm the strengths of the new approach on standard machine learning problems. |
Year | Venue | DocType |
|---|---|---|
2021 | 24TH INTERNATIONAL CONFERENCE ON ARTIFICIAL INTELLIGENCE AND STATISTICS (AISTATS) | Conference |
Volume | ISSN | Citations |
130 | 2640-3498 | 0 |
PageRank | References | Authors |
0.34 | 0 | 5 |
Name | Order | Citations | PageRank |
|---|---|---|---|
| Jahani Majid | 1 | 0 | 0.68 |
| Nazari Mohammadreza | 2 | 0 | 0.34 |
| Rachel Tappenden | 3 | 68 | 5.56 |
| Albert S. Berahas | 4 | 21 | 4.05 |
| Martin Takác | 5 | 752 | 49.49 |