Title | ||
---|---|---|
Combining self-organizing maps with mixtures of experts: application to an actor-critic model of reinforcement learning in the basal ganglia |
Abstract | ||
---|---|---|
In a reward-seeking task performed in a continuous environment, our
previous work compared several {Actor-Critic} {(AC)} architectures
implementing dopamine-like reinforcement learning mechanisms in the
rat’s basal ganglia. The task complexity imposes the coordination
of several {AC} submodules, each module being an expert trained in
a particular subset of the task. We showed that the classical method
where the choice of the expert to train at a given time depends on
each expert’s performance suffered from strong limitations. We rather
proposed to cluster the continuous state space by an ad hoc method
that lacked autonomy and generalization abilities. In the present
work we have combined the mixture of experts with self-organizing
maps in order to cluster autonomously the experts’ responsibility
space. On the one hand, we find that classical Kohonen maps give
very variable results: some task decompositions provide very good
and stable reinforcement learning performances, whereas some others
are unadapted to the task. Moreover, they require the number of experts
to be set a priori. On the other hand, algorithms like Growing Neural
Gas or Growing When Required have the property to choose autonomously
and incrementally the number of experts to train. They lead to good
performances, even if they are still weaker than our hand-tuned task
decomposition and than the best Kohonen maps that we got. We finally
discuss on propositions about what information to add to these algorithms,
such as knowledge of current behavior, in order to make the task
decomposition appropriate to the reinforcement learning process. |
Year | DOI | Venue |
---|---|---|
2006 | 10.1007/11840541_33 | Simulation of Adaptive Behavior |
Keywords | Field | DocType |
classical method,self-organizing map,ac submodules,stable reinforcement,task complexity,kohonen map,actor-critic model,hand-tuned task decomposition,classical kohonen map,reward-seeking task,dopamine-like reinforcement,task decomposition,basal ganglion,dopamine,reinforcement learning,state space | Virtual reality,Computer science,A priori and a posteriori,Self-organization,Self-organizing map,Artificial intelligence,Artificial neural network,State space,Neural gas,Machine learning,Reinforcement learning | Conference |
Volume | ISSN | ISBN |
4095 | 0302-9743 | 3-540-38608-4 |
Citations | PageRank | References |
7 | 0.63 | 13 |
Authors | ||
3 |
Name | Order | Citations | PageRank |
---|---|---|---|
Khamassi Mehdi | 1 | 112 | 16.51 |
Louis-Emmanuel Martinet | 2 | 20 | 2.88 |
Agnè/s Guillot | 3 | 27 | 1.86 |