Title | ||
---|---|---|
A Monte Carlo Neural Fictitious Self-Play Approach To Approximate Nash Equilibrium In Imperfect-Information Dynamic Games |
Abstract | ||
---|---|---|
Solving the optimization problem to approach a Nash Equilibrium point plays an important role in imperfect information games, e.g., StarCraft and poker. Neural Fictitious Self-Play (NFSP) is an effective algorithm that learns approximate Nash Equilibrium of imperfect-information games from purely self-play without prior domain knowledge. However, it needs to train a neural network in an off-policy manner to approximate the action values. For games with large search spaces, the training may suffer from unnecessary exploration and sometimes fails to converge. In this paper, we propose a new Neural Fictitious Self-Play algorithm that combines Monte Carlo tree search with NFSP, called MC-NFSP, to improve the performance in real-time zero-sum imperfect-information games. With experiments and empirical analysis, we demonstrate that the proposed MC-NFSP algorithm can approximate Nash Equilibrium in games with large-scale search depth while the NFSP can not. Furthermore, we develop an Asynchronous Neural Fictitious Self-Play framework (ANFSP). It uses asynchronous and parallel architecture to collect game experience and improve both the training efficiency and policy quality. The experiments with th e games with hidden state information (Texas Hold'em), and the FPS (firstperson shooter) games demonstrate effectiveness of our algorithms. |
Year | DOI | Venue |
---|---|---|
2021 | 10.1007/s11704-020-9307-6 | FRONTIERS OF COMPUTER SCIENCE |
Keywords | DocType | Volume |
approximate Nash Equilibrium, imperfect-information games, dynamic games, Monte Carlo tree search, Neural Fictitious Self-Play, reinforcement learning | Journal | 15 |
Issue | ISSN | Citations |
5 | 2095-2228 | 0 |
PageRank | References | Authors |
0.34 | 0 | 7 |
Name | Order | Citations | PageRank |
---|---|---|---|
Li Zhang | 1 | 141 | 20.37 |
Yuxuan Chen | 2 | 5 | 8.88 |
Wei Wang | 3 | 6 | 1.14 |
Ziliang Han | 4 | 0 | 0.34 |
Shijian Li | 5 | 1155 | 69.34 |
Zhijie Pan | 6 | 0 | 0.34 |
Gang Pan | 7 | 0 | 1.35 |