Paper Info
Title
A Friend-or-Foe framework for multi-agent reinforcement learning policy generation in mixing cooperative-competitive scenarios
Abstract
Although multi-agent deep deterministic policy gradient is a classic deep reinforcement learning algorithm in multi-agent systems. It also has critical problems such as poor training stability and low policy robustness, which significantly limit the capability and application of the algorithm. So this article proposes an improved algorithm called friend-or-foe multi-agent deep deterministic policy gradient for solving the above problems. The main innovations are as follows: (1) inspired by the concept of friend-or-foe game theory, we modified the framework of the original multi-agent deep deterministic policy gradient by using two identical training networks with agents' optimal and worst actions input, which improves the robustness of training policies, and (2) we propose an action perturbation technique based on gradient-descent to expand the selection range of actions, thereby improving training stability of our proposing algorithm. Finally, we conducted multiple sets of comparative experiments between our friend-or-foe multi-agent deep deterministic policy gradient and original one in four authoritative mixed cooperative-competitive scenarios. The results show that our improving algorithm can simultaneously improve the training stability and the robustness of agents' generating policies in different complicated environments.
Year
DOI
Venue
2022
10.1177/01423312221077755
TRANSACTIONS OF THE INSTITUTE OF MEASUREMENT AND CONTROL
Keywords
DocType
Volume
Deep reinforcement learning, multi-agent reinforcement learning, multi-agent system game theory
Journal
44
Issue
ISSN
Citations 
12
0142-3312
0
PageRank 
References 
Authors
0.34
0
7
Name
Order
Citations
PageRank
Yu Sun120835.82
Jun Lai200.34
Lei Cao310.71
Xiliang Chen400.34
Zhixiong Xu500.34
Zhen Lian600.34
Huijin Fan700.34