Paper Info
Title
Explore Deep Neural Network And Reinforcement Learning To Large-Scale Tasks Processing In Big Data
Abstract
Large-scale tasks processing based on cloud computing has become crucial to big data analysis and disposal in recent years. Most previous work, generally, utilize the conventional methods and architectures for general scale tasks to achieve tons of tasks disposing, which is limited by the issues of computing capability, data transmission, etc. Based on this argument, a fat-tree structure-based approach called LTDR (Large-scale Tasks processing using Deep network model and Reinforcement learning) has been proposed in this work. Aiming at exploring the optimal task allocation scheme, a virtual network mapping algorithm based on deep convolutional neural network and Q-learning is presented herein. After feature extraction, we design and implement a policy network to make node mapping decisions. The link mapping scheme can be attained by the designed distributed value-function based reinforcement learning model. Eventually, tasks are allocated onto proper physical nodes and processed efficiently. Experimental results show that LTDR can significantly improve the utilization of physical resources and long-term revenue while satisfying task requirements in big data.
Year
DOI
Venue
2019
10.1142/S0218001419510108
INTERNATIONAL JOURNAL OF PATTERN RECOGNITION AND ARTIFICIAL INTELLIGENCE
Keywords
Field
DocType
Big data, cloud computing, deep network model, large-scale tasks processing, reinforcement learning
Artificial intelligence,Artificial neural network,Big data,Machine learning,Mathematics,Reinforcement learning,Cloud computing
Journal
Volume
Issue
ISSN
33
13
0218-0014
Citations 
PageRank 
References 
0
0.34
13
Authors
4
Name
Order
Citations
PageRank
Chunyi Wu152.43
Gaochao Xu218324.11
Yan Ding34012.03
Jia Zhao4404.84