Name
Title | ||
|---|---|---|
A Unified Switching System Perspective and Convergence Analysis of Q-Learning Algorithms. |
Abstract | ||
|---|---|---|
This paper develops a novel and unified framework to analyze the convergence of a large family of Q-learning algorithms from the switching system perspective. We show that the nonlinear ODE models associated with Q-learning and many of its variants can be naturally formulated as affine switching systems. Building on their asymptotic stability, we obtain a number of interesting results: (i) we provide a simple ODE analysis for the convergence of asynchronous Q-learning under relatively weak assumptions; (ii) we establish the first convergence analysis of the averaging Q-learning algorithm; and (iii) we derive a new sufficient condition for the convergence of Q-learning with linear function approximation. |
Year | Venue | DocType |
|---|---|---|
2020 | NeurIPS | Conference |
Volume | Citations | PageRank |
33 | 0 | 0.34 |
References | Authors | |
0 | 2 | |
Name | Order | Citations | PageRank |
|---|---|---|---|
| Donghwan Lee | 1 | 25 | 9.30 |
| Niao He | 2 | 212 | 16.52 |