Paper Info
Title
Cooperative traffic signal control using Multi-step return and Off-policy Asynchronous Advantage Actor-Critic Graph algorithm.
Abstract
Intelligent traffic signal control helps to reduce traffic congestion and thus has been studied for a few decades. Multi-intersection cooperative traffic signal control (CTSC), which is more practical than single-intersection traffic signal control, has attracted much attention and research in recent years. Existing works on multi-intersection CTSC make responsive policies based on the sequence of agents’ actions. One issue in multi-intersection CTSC is that every agent’s actions are mapped from its own road information and some useful information, e.g., the distance of adjacent agents, is ignored, which may lead to suboptimal traffic signal control policies. To address this issue, in this paper a decentralized coordination graph algorithm, referred to as Multi-step return and Off-policy Asynchronous Advantage Actor-Critic Graph (MOA3CG) algorithm, is proposed. The MOA3CG algorithm is based on an asynchronous method of multiagent deep reinforcement learning and a coordination graph; the proposed algorithm makes traffic signal control policies based on current traffic states, the history of observations and other information. A new reward function and An Adjusting Matrix of Traffic Signal Phase Control (AMTSPC) are proposed, which are used by the MOA3CG algorithm in the policy-making process; the AMTSPC is to alter selection of actions by considering the distance of adjacent agents. Experimental results on real-world road scenarios show that the proposed algorithm outperforms other four state-of-the-art algorithms in terms of average delay, average traveling time of vehicles, and the throughput of vehicles, thus eventually helps to mitigate traffic congestion.
Year
DOI
Venue
2019
10.1016/j.knosys.2019.07.026
Knowledge-Based Systems
Keywords
Field
DocType
Cooperative traffic signal control,Coordination graph algorithm,Multiagent deep reinforcement learning,Transfer planning,Asynchronous Advantage Actor-Critic (A3C) algorithm
Graph algorithms,Asynchronous communication,Data mining,Traffic signal,Matrix (mathematics),Computer science,Computer network,Throughput,Asynchronous method invocation,Traffic congestion,Reinforcement learning
Journal
Volume
ISSN
Citations 
183
0950-7051
3
PageRank 
References 
Authors
0.38
0
4
Name
Order
Citations
PageRank
Shantian Yang141.75
Bo Yang251952.33
Hau-San Wong3100886.89
Zhongfeng Kang432.75