Paper Info
Title
Linear Feature Encoding for Reinforcement Learning.
Abstract
Feature construction is of vital importance in reinforcement learning, as the quality of a value function or policy is largely determined by the corresponding features. The recent successes of deep reinforcement learning (RL) only increase the importance of understanding feature construction. Typical deep RL approaches use a linear output layer, which means that deep RL can be interpreted as a feature construction/encoding network followed by linear value function approximation. This paper develops and evaluates a theory of linear feature encoding. We extend theoretical results on feature quality for linear value function approximation from the uncontrolled case to the controlled case. We then develop a supervised linear feature encoding method that is motivated by insights from linear value function approximation theory, as well as empirical successes from deep RL. The resulting encoder is a surprisingly effective method for linear value function approximation using raw images as inputs.
Year
Venue
Field
2016
ADVANCES IN NEURAL INFORMATION PROCESSING SYSTEMS 29 (NIPS 2016)
Computer science,Effective method,Approximation theory,Bellman equation,Artificial intelligence,Encoder,Machine learning,Encoding (memory),Reinforcement learning
DocType
Volume
ISSN
Conference
29
1049-5258
Citations 
PageRank 
References 
0
0.34
0
Authors
4
Name
Order
Citations
PageRank
Zhao Song117725.62
Ronald Parr22428186.85
Xuejun Liao371143.68
L. Carin44603339.36