Title
Deep Reinforcement Learning for Cooperative Content Caching in Vehicular Edge Computing and Networks
Abstract
In this article, we propose a cooperative edge caching scheme, a new paradigm to jointly optimize the content placement and content delivery in the vehicular edge computing and networks, with the aid of the flexible trilateral cooperations among a macro-cell station, roadside units, and smart vehicles. We formulate the joint optimization problem as a double time-scale Markov decision process (DTS-MDP), based on the fact that the time-scale of content timeliness changes less frequently as compared to the vehicle mobility and network states during the content delivery process. At the beginning of the large time-scale, the content placement/updating decision can be obtained according to the content popularity, vehicle driving paths, and resource availability. On the small time-scale, the joint vehicle scheduling and bandwidth allocation scheme is designed to minimize the content access cost while satisfying the constraint on content delivery latency. To solve the long-term mixed integer linear programming (LT-MILP) problem, we propose a nature-inspired method based on the deep deterministic policy gradient (DDPG) framework to obtain a suboptimal solution with a low computation complexity. The simulation results demonstrate that the proposed cooperative caching system can reduce the system cost, as well as the content delivery latency, and improve content hit ratio, as compared to the noncooperative and random edge caching schemes.
Year
DOI
Venue
2020
10.1109/JIOT.2019.2945640
IEEE Internet of Things Journal
Keywords
Field
DocType
Cooperative caching,Optimization,Edge computing,Computational modeling,Internet of Things,Indexes,Base stations
Vehicular edge computing,Scheduling (computing),Computer science,Latency (engineering),Bandwidth allocation,Markov decision process,Computer network,Integer programming,Optimization problem,Reinforcement learning,Distributed computing
Journal
Volume
Issue
ISSN
7
1
2327-4662
Citations 
PageRank 
References 
25
0.69
0
Authors
5
Name
Order
Citations
PageRank
Guanhua Qiao1795.29
Supeng Leng270267.45
Sabita Maharjan3107852.89
Yan Zhang45818354.13
Nirwan Ansari54667357.64