Paper Info
Title
Hierarchical Reinforcement Learning for Blockchain-Assisted Software Defined Industrial Energy Market
Abstract
Energy Internet (EI) is developing and booming rapidly with the increase of distributed energy resources, which is beneficial to address the severe condition of industrial energy. However, there are inevitable credit crises and utility optimization challenges in EI that need to be settled. In this article, we propose a blockchain-assisted software defined energy Internet (BSDEI), where a distributed energy market smart contract is designed to ensure transactions executed reliably and participants’ accounts dealt accurately. In order to jointly optimize the utilities of operators, retailers, and industrial prosumers in BSDEI, we formulate the whole trading process as a three-stage Stackelberg game, with the proof of existence and uniqueness for the Stackelberg equilibrium. Then, we design a hierarchical distributed policy gradient algorithm to solve the Stackelberg game under incomplete information. We implement a blockchain-based industrial energy trading system using a middleware platform. The smart contract is deployed on the consortium blockchain, providing website interfaces for participants to operate. Furthermore, we conduct experiments for analyzing economic benefits. Our system prototype demonstrates the feasibility of BSDEI and the algorithm exceeds about 18% in total mean reward than comparing algorithms.
Year
DOI
Venue
2022
10.1109/TII.2022.3140878
IEEE Transactions on Industrial Informatics
Keywords
DocType
Volume
Blockchain,industrial energy market,reinforcement learning,software defined network (SDN),Stackelberg game
Journal
18
Issue
ISSN
Citations 
9
1551-3203
0
PageRank 
References 
Authors
0.34
18
4
Name
Order
Citations
PageRank
Yifan Cao141.07
Xiaoxu Ren241.74
Chao Qiu3143.89
Xiaofei Wang468658.88