Name
Title | ||
|---|---|---|
Accelerated Proximal Alternating Gradient-Descent-Ascent for Nonconvex Minimax Machine Learning |
Abstract | ||
|---|---|---|
Alternating gradient-descent-ascent (AltGDA) is an optimization algorithm that has been widely used for model training in various machine learning applications, which aims to solve a nonconvex minimax optimization problem. However, the existing studies show that it suffers from a high computation complexity in nonconvex minimax optimization. In this paper, we develop a single-loop and fast AltGDA-type algorithm that leverages proximal gradient updates and momentum acceleration to solve regularized nonconvex minimax optimization problems. By leveraging the momentum acceleration technique, we prove that the algorithm converges to a critical point in nonconvex minimax optimization and achieves a computation complexity in the order of $\mathcal{O}\left( {{\kappa ^{\frac{{11}}{6}}}{\varepsilon ^{ - 2}}} \right)$, where ϵ is the desired level of accuracy and κ is the problem’s condition number. Such a computation complexity improves the state-of-the-art complexities of single-loop GDA and AltGDA algorithms (see the summary of comparison in Table I). We demonstrate the effectiveness of our algorithm via an experiment on adversarial deep learning. |
Year | DOI | Venue |
|---|---|---|
2022 | 10.1109/ISIT50566.2022.9834691 | 2022 IEEE International Symposium on Information Theory (ISIT) |
Keywords | DocType | ISSN |
Minimiax optimization,alternating gradient descent ascent,proximal gradient,momentum,complexity | Conference | 2157-8095 |
ISBN | Citations | PageRank |
978-1-6654-2160-7 | 0 | 0.34 |
References | Authors | |
8 | 3 | |
Name | Order | Citations | PageRank |
|---|---|---|---|
| Ziyi Chen | 1 | 0 | 0.34 |
| Shaocong Ma | 2 | 0 | 1.69 |
| Yi Zhou | 3 | 65 | 17.55 |