Paper Info
Title
A Turbo Q-Learning (Tql) For Energy Efficiency Optimization In Heterogeneous Networks
Abstract
In order to maximize energy efficiency in heterogeneous networks (HetNets), a turbo Q-Learning (TQL) combined with multistage decision process and tabular Q-Learning is proposed to optimize the resource configuration. For the large dimensions of action space, the problem of energy efficiency optimization is designed as a multistage decision process in this paper, according to the resource allocation of optimization objectives, the initial problem is divided into several subproblems which are solved by tabular Q-Learning, and the traditional exponential increasing size of action space is decomposed into linear increase. By iterating the solutions of subproblems, the initial problem is solved. The simple stability analysis of the algorithm is given in this paper. As to the large dimension of state space, we use a deep neural network (DNN) to classify states where the optimization policy of novel Q-Learning is set to label samples. Thus far, the dimensions of action and state space have been solved. The simulation results show that our approach is convergent, improves the convergence speed by 60% while maintaining almost the same energy efficiency and having the characteristics of system adjustment.
Year
DOI
Venue
2020
10.3390/e22090957
ENTROPY
Keywords
DocType
Volume
energy efficiency, HetNets, eICIC, Q-Learning, reinforcement learning, multistage decision process
Journal
22
Issue
ISSN
Citations 
9
1099-4300
0
PageRank 
References 
Authors
0.34
0
4
Name
Order
Citations
PageRank
Xiumin Wang198.85
Lei Li201.35
li jun39342.84
Zheng-quan Li43013.94