Paper Info
Title
Learning model-based planning from scratch.
Abstract
Conventional wisdom holds that model-based planning is a powerful approach to sequential decision-making. It is often very challenging in practice, however, because while a model can be used to evaluate a plan, it does not prescribe how to construct a plan. Here we introduce the Imagination-based Planner, the first model-based, sequential decision-making agent that can learn to construct, evaluate, and execute plans. Before any action, it can perform a variable number of steps, which involve proposing an imagined action and evaluating it with its model-based imagination. All imagined actions and outcomes are aggregated, iteratively, into a context which conditions future real and imagined actions. The agent can even decide how to imagine: testing out alternative imagined actions, chaining sequences of actions together, or building a more complex imagination tree by navigating flexibly among the previously imagined states using a learned policy. And our agent can learn to plan economically, jointly optimizing for external rewards and computational costs associated with using its imagination. We show that our architecture can learn to solve a challenging continuous control problem, and also learn elaborate planning strategies in a discrete maze-solving task. Our work opens a new direction toward learning the components of a model-based planning system and how to use them.
Year
Venue
Field
2017
arXiv: Artificial Intelligence
Architecture,Chaining,Scratch,Computer science,Planner,Conventional wisdom,Artificial intelligence,Imagination,Machine learning
DocType
Volume
Citations 
Journal
abs/1707.06170
16
PageRank 
References 
Authors
0.62
13
10
Name
Order
Citations
PageRank
Razvan Pascanu12596199.21
Yujia Li240523.01
Oriol Vinyals39419418.45
Nicolas Heess4176294.77
Lars Buesing524816.50
Sébastien Racanière6281.42
David P. Reichert7886.85
Theophane Weber815916.79
Daan Wierstra95412255.92
Peter Battaglia1044222.63