Name
Abstract | ||
|---|---|---|
In this paper we address the combination of batch reinforcement-learning (BRL) techniques with direct policy search (DPS) algorithms in the context of robot learning. Batch value-based algorithms (such as fitted Q-iteration) have been proved to outperform online ones in many complex applications, but they share the same difficulties in solving problems with continuous action spaces, such as robotic ones. In these cases, actor-critic and DPS methods are preferable, since the optimization process is limited to a family of parameterized (usually smooth) policies. On the other hand, these methods (e.g., policy gradient and evolutionary methods) are generally very expensive, since finding the optimal parameterization may require to evaluate the performance of several policies, which in many real robotic applications is unfeasible or even dangerous. To overcome such problems, we exploit the fitted policy search (FPS) approach, in which the expected return of any policy considered during the optimization process is evaluated offline (without resorting to the robot) by reusing the data collected in the initial exploration phase. In this way, it is possible to take the advantages of both BRL and DPS algorithms, thus achieving an effective learning approach to solve robotic problems. A balancing task on a real two-wheeled robotic pendulum is used to analyze the properties and evaluate the effectiveness of the FPS approach. |
Year | DOI | Venue |
|---|---|---|
2011 | 10.1109/ADPRL.2011.5967368 | ADPRL |
Keywords | Field | DocType |
optimisation,robotic applications,optimization process,batch reinforcement learning techniques,learning (artificial intelligence),robot learning,direct policy search algorithms,mobile robots,search problems,pendulums,continuous action spaces,two-wheeled robotic pendulum,fitted policy search,batch value-based algorithms,robot kinematics,data collection,approximation algorithms,reinforcement learning,learning artificial intelligence,optimization | Robot learning,Parameterized complexity,Mathematical optimization,Reuse,Computer science,Exploit,Artificial intelligence,Pendulum,Robot,Expected return,Mobile robot,Machine learning | Conference |
ISBN | Citations | PageRank |
978-1-4244-9887-1 | 0 | 0.34 |
References | Authors | |
0 | 5 | |
Name | Order | Citations | PageRank |
|---|---|---|---|
| Martino Migliavacca | 1 | 10 | 2.82 |
| Alessio Pecorino | 2 | 1 | 0.77 |
| Matteo Pirotta | 3 | 78 | 18.50 |
| Marcello Restelli | 4 | 416 | 61.31 |
| Andrea Bonarini | 5 | 623 | 76.73 |