Title
Adaptive co-construction of state and action spaces in reinforcement learning
Abstract
Reinforcement learning (RL) attracts much attention as a technique for realizing computational intelligence such as adaptive and autonomous decentralized systems. In general, however, it is not easy to put RL to practical use. The difficulty includes the problem of designing suitable state and action spaces for an agent. Previously, we proposed an adaptive state space construction method which is called a "state space filter," and an adaptive action space construction method which is called "switching RL," after the other space has been fixed. In this article, we reconstitute these two construction methods as one method by treating the former and the latter as a combined method for mimicking an infant's perceptual development. In this method, perceptual differentiation progresses as an infant become older and more experienced, and the infant's motor development, in which gross motor skills develop before fine motor skills, also progresses. The proposed method is based on introducing and referring to "entropy." In addition, a computational experiment was conducted using a so-called "path planning problem" with continuous state and action spaces. As a result, the validity of the proposed method has been confirmed.
Year
DOI
Venue
2011
10.1007/s10015-011-0883-2
Artificial Life and Robotics
Keywords
DocType
Volume
adaptive state space construction,action space,adaptive action space construction,continuous state,suitable state,construction method,combined method,state space filter,fine motor skill,reinforcement learning,adaptive co-construction,reinforcement learning rl · state and action spaces design · q-learning · actor-critic · entropy,motor development,q learning,computational intelligence,state space,decentralized system,computer experiment,motor skills,path planning,entropy
Journal
16
Issue
ISSN
Citations 
1
1614-7456
0
PageRank 
References 
Authors
0.34
0
3
Name
Order
Citations
PageRank
Masato Nagayoshi131.89
Hajime Murao2216.70
Hisashi Tamaki314140.54