Paper Info
Title
Hierarchical Decision and Control for Continuous Multitarget Problem: Policy Evaluation With Action Delay.
Abstract
This paper proposes a hierarchical decision-making and control algorithm for the shepherd game, the seventh mission in the International Aerial Robotics Competition (IARC). In this game, the agent (a multirotor aerial robot) is required to contact targets (ground vehicles) sequentially and drive them to a certain boundary to earn score. During the game of 10 min, the agent should be fully autonomous without any human interference. Regarding the lower-level controller and dynamics of the agent, each action takes a duration of time to accomplish. Denoted as an action delay, in this paper, this action duration is nonconstant and is related to the final reward. Therefore, the challenging point is making the agent ``aware of time'' when applying a certain action. We solve this problem by two approaches: deep Q-networks and lookup table. The action delay predictor in the decision-level is fitted by a lower-level controller. Through simulations by the example of the shepherd game, the effectiveness and efficiency of this approach are validated. This paper helps our team winning the first prize in IARC 2017, and keeps the best record of this mission since it was released in 2013.
Year
DOI
Venue
2019
10.1109/TNNLS.2018.2844466
IEEE transactions on neural networks and learning systems
Keywords
Field
DocType
Games,Mobile robots,Delays,Drones,Decision making,Collision avoidance
Lookup table,Control theory,Computer science,Artificial intelligence,Interference (wave propagation),Drone,Robot,Multirotor,Machine learning,Mobile robot,Robotics
Journal
Volume
Issue
ISSN
30
2
2162-2388
Citations 
PageRank 
References 
0
0.34
8
Authors
5
Name
Order
Citations
PageRank
Jiangcheng Zhu101.01
Jun Zhu2153.61
Zhepei Wang3105.61
Shan Guo400.34
Chao Xu513633.20