Name
Abstract | ||
|---|---|---|
Reinforcement learning can train policies that effectively perform complex tasks. However for long-horizon tasks, the performance of these methods degrades with horizon, often necessitating reasoning over and composing lower-level skills. Hierarchical reinforcement learning aims to enable this by providing a bank of low-level skills as action abstractions. Hierarchies can further improve on this by abstracting the space states as well. We posit that a suitable state abstraction should depend on the capabilities of the available lower-level policies. We propose Value Function Spaces: a simple approach that produces such a representation by using the value functions corresponding to each lower-level skill. These value functions capture the affordances of the scene, thus forming a representation that compactly abstracts task relevant information and robustly ignores distractors. Empirical evaluations for maze-solving and robotic manipulation tasks demonstrate that our approach improves long-horizon performance and enables better zero-shot generalization than alternative model-free and model-based methods. |
Year | Venue | Keywords |
|---|---|---|
2022 | International Conference on Learning Representations (ICLR) | hierarchical reinforcement learning,planning,representation learning,robotics |
DocType | Citations | PageRank |
Conference | 0 | 0.34 |
References | Authors | |
0 | 7 | |
Name | Order | Citations | PageRank |
|---|---|---|---|
| Dhruv Shah | 1 | 0 | 0.34 |
| Peng Xu | 2 | 0 | 0.34 |
| Yao Lu | 3 | 0 | 0.34 |
| Ted Xiao | 4 | 14 | 2.31 |
| Alexander Toshev | 5 | 0 | 0.34 |
| Sergey Levine | 6 | 3377 | 182.21 |
| Brian Ichter | 7 | 0 | 0.34 |