Paper Info
Title
OnRL: Improving Mobile Video Telephony via Online Reinforcement Learning
Abstract
Machine learning models, particularly reinforcement learning (RL), have demonstrated great potential in optimizing video streaming applications. However, the state-of-the-art solutions are limited to an "offline learning" paradigm, i.e., the RL models are trained in simulators and then are operated in real networks. As a result, they inevitably suffer from the simulation-to-reality gap, showing far less satisfactory performance under real conditions compared with simulated environment. In this work, we close the gap by proposing OnRL, an online RL framework for real-time mobile video telephony. OnRL puts many individual RL agents directly into the video telephony system, which make video bitrate decisions in real-time and evolve their models over time. OnRL then aggregates these agents to form a high-level RL model that can help each individual to react to unseen network conditions. Moreover, OnRL incorporates novel mechanisms to handle the adverse impacts of inherent video traffic dynamics, and to eliminate risks of quality degradation caused by the RL model's exploration attempts. We implement OnRL on a mainstream operational video telephony system, Alibaba Taobao-live. In a month-long evaluation with 543 hours of video sessions from 151 real-world mobile users, OnRL outperforms the prior algorithms significantly, reducing video stalling rate by 14.22% while maintaining similar video quality.
Year
DOI
Venue
2020
10.1145/3372224.3419186
MobiCom '20: The 26th Annual International Conference on Mobile Computing and Networking London United Kingdom September, 2020
DocType
ISBN
Citations 
Conference
978-1-4503-7085-1
2
PageRank 
References 
Authors
0.36
21
9
Name
Order
Citations
PageRank
Zhang Huaguang136224.33
Aoying Zhou22632238.85
J. Lu320.36
Ronghua Ma411324.96
Yihua Hu58426.61
Chengrong Li6699.58
Xinyu Zhang7134378.62
Huadong Ma82020179.93
X. Chen920.36