Title
Bifurcation analysis of reinforcement learning agents in the Selten's horse game
Abstract
The application of reinforcement learning algorithms to multiagent domains may cause complex non-convergent dynamics. The replicator dynamics, commonly used in evolutionary game theory, proved to be effective for modeling the learning dynamics in normal form games. Nonetheless, it is often interesting to study the robustness of the learning dynamics when either learning or structural parameters are perturbed. This is equivalent to unfold the catalog of learning dynamical scenarios that arise for all possible parameter settings which, unfortunately, cannot be obtained through ``brute force'' simulation of the replicator dynamics. The analysis of bifurcations, i.e., critical parameter combinations at which the learning behavior undergoes radical changes, is mandatory. In this work, we introduce a one-parameter bifurcation analysis of the Selten's Horse game in which the learning process exhibits a set of complex dynamical scenarios even for relatively small perturbations on payoffs.
Year
DOI
Venue
2007
10.1007/978-3-540-77949-0_10
Autonomous Agents and Multi-Agent Systems
Keywords
Field
DocType
critical parameter combination,complex non-convergent dynamic,dynamical scenario,replicator dynamic,normal form game,horse game,possible parameter setting,one-parameter bifurcation analysis,complex dynamical scenario,evolutionary game theory
Bifurcation analysis,Computer science,Replicator equation,Theoretical computer science,Robustness (computer science),Brute force,Artificial intelligence,Evolutionary game theory,Nash equilibrium,Hopf bifurcation,Reinforcement learning
Conference
ISBN
Citations 
PageRank 
3-540-77947-7
0
0.34
References 
Authors
4
4
Name
Order
Citations
PageRank
Alessandro Lazaric151848.19
Enrique Munoz de Cote217420.97
Fabio Dercole34714.32
Marcello Restelli441661.31