Name
Abstract | ||
|---|---|---|
Both global methods and on-line trajectory optimization methods are powerful techniques for solving optimal control problems; however, each has limitations. In order to mitigate the undesirable properties of each, we explore the possibility of combining the two. We explore two methods of deriving a descriptive final cost function to assist model predictive control (MPC) in selecting a good policy without having to plan as far into the future or having to fine-tune delicate cost functions. First, we exploit the large amount of data which is generated in MPC simulations (based on the receding horizon iterative LQG method) to learn, off-line, the global optimal value function for use as a final cost. We demonstrate that, while the global function approximation matches the value function well on some problems, there is relatively little improvement to the original MPC. Alternatively, we solve the Bellman equation directly using aggregation methods for linearly-solvable Markov Decision Processes to obtain an approximation to the value function and the optimal policy. Using both pieces of information in the MPC framework, we find controller performance of similar quality to MPC alone with long horizon, but now we may drastically shorten the horizon. Implementation of these methods shows that Bellman equation-based methods and on-line trajectory methods can be combined in real applications to the benefit of both. |
Year | DOI | Venue |
|---|---|---|
2013 | 10.1109/ADPRL.2013.6614995 | PROCEEDINGS OF THE 2013 IEEE SYMPOSIUM ON ADAPTIVE DYNAMIC PROGRAMMING AND REINFORCEMENT LEARNING (ADPRL) |
Keywords | Field | DocType |
iterative methods,polynomials,function approximation,trajectory,mathematical model,model predictive control,markov processes,learning artificial intelligence,optimization,linear quadratic gaussian control,predictive control | Mathematical optimization,Optimal control,Function approximation,Trajectory optimization,Linear-quadratic-Gaussian control,Iterative method,Control theory,Model predictive control,Markov decision process,Bellman equation,Mathematics | Conference |
ISSN | Citations | PageRank |
2325-1824 | 6 | 0.70 |
References | Authors | |
10 | 5 | |
Name | Order | Citations | PageRank |
|---|---|---|---|
| Mingyuan Zhong | 1 | 6 | 0.70 |
| M. Johnson | 2 | 6 | 0.70 |
| Yuval Tassa | 3 | 1097 | 52.33 |
| Tom Erez | 4 | 1027 | 50.56 |
| Emanuel Todorov | 5 | 1591 | 126.81 |