Name
Abstract | ||
|---|---|---|
Projecting high-dimensional environment observations into lower-dimensional structured representations can considerably improve data-efficiency for reinforcement learning in domains with limited data such as robotics. Can a single generally useful representation be found? In order to answer this question, it is important to understand how the representation will be used by the agent and what properties such a good representation should have. In this paper we systematically evaluate a number of common learnt and hand-engineered representations in the context of three robotics tasks: lifting, stacking and pushing of 3D blocks. The representations are evaluated in two use-cases: as input to the agent, or as a source of auxiliary tasks. Furthermore, the value of each representation is evaluated in terms of three properties: dimensionality, observability and disentanglement. We can significantly improve performance in both use-cases and demonstrate that some representations can perform commensurate to simulator states as agent inputs. Finally, our results challenge common intuitions by demonstrating that: 1) dimensionality strongly matters for task generation, but is negligible for inputs, 2) observability of task-relevant aspects mostly affects the input representation use-case, and 3) disentanglement leads to better auxiliary tasks, but has only limited benefits for input representations. This work serves as a step towards a more systematic understanding of what makes a good representation for control in robotics, enabling practitioners to make more informed choices for developing new learned or hand-engineered representations. |
Year | DOI | Venue |
|---|---|---|
2021 | 10.1109/ICRA48506.2021.9560733 | 2021 IEEE INTERNATIONAL CONFERENCE ON ROBOTICS AND AUTOMATION (ICRA 2021) |
DocType | Volume | Issue |
Conference | 2021 | 1 |
ISSN | Citations | PageRank |
1050-4729 | 0 | 0.34 |
References | Authors | |
4 | 11 | |
Name | Order | Citations | PageRank |
|---|---|---|---|
| markus wulfmeier | 1 | 51 | 6.86 |
| Arunkumar Byravan | 2 | 78 | 5.56 |
| Tim Hertweck | 3 | 0 | 1.35 |
| Irina Higgins | 4 | 245 | 11.95 |
| Ankush Gupta | 5 | 137 | 5.99 |
| Tejas D. Kulkarni | 6 | 414 | 19.36 |
| Malcolm Reynolds | 7 | 100 | 4.30 |
| Denis Teplyashin | 8 | 43 | 2.89 |
| Roland Hafner | 9 | 22 | 2.70 |
| Thomas Lampe | 10 | 21 | 2.33 |
| Martin Riedmiller | 11 | 5655 | 366.29 |