Name
Abstract | ||
|---|---|---|
In deterministic systems, reinforcement learning-based online approximate optimal control methods typically require a restrictive persistence of excitation (PE) condition for convergence. This paper presents a concurrent learning-based solution to the online approximate optimal regulation problem that eliminates the need for PE. The development is based on the observation that given a model of the system, the Bellman error, which quantifies the deviation of the system Hamiltonian from the optimal Hamiltonian, can be evaluated at any point in the state space. Further, a concurrent learning-based parameter identifier is developed to compensate for parametric uncertainty in the plant dynamics. Uniformly ultimately bounded (UUB) convergence of the system states to the origin, and UUB convergence of the developed policy to an approximate optimal policy are established using a Lyapunov-based analysis, and simulations are performed to demonstrate the performance of the developed controller. |
Year | DOI | Venue |
|---|---|---|
2013 | 10.1109/CDC.2013.6760878 | conference on decision and control |
Keywords | Field | DocType |
Lyapunov methods,approximation theory,compensation,convergence,learning (artificial intelligence),optimal control,uncertain systems,Bellman error,Lyapunov-based analysis,PE condition,UUB convergence,approximate optimal policy,concurrent learning,deterministic systems,online approximate optimal control methods,online approximate optimal regulation problem,optimal Hamiltonian,parameter identifier,parametric uncertainty compensation,plant dynamics,reinforcement learning,restrictive persistence of excitation condition,state space,uniformly ultimately bounded convergence | Convergence (routing),Lyapunov function,Mathematical optimization,Control theory,Optimal control,Control theory,Computer science,Parametric statistics,State space,Bounded function,Reinforcement learning | Journal |
Volume | ISSN | ISBN |
abs/1304.3477 | 0743-1546 | 978-1-4673-5714-2 |
Citations | PageRank | References |
12 | 0.76 | 8 |
Authors | ||
3 | ||
Name | Order | Citations | PageRank |
|---|---|---|---|
| R. Kamalapurkar | 1 | 367 | 26.14 |
| Patrick Walters | 2 | 81 | 4.74 |
| Warren E. Dixon | 3 | 1757 | 149.42 |