Title
Verifying Deep-RL-Driven Systems.
Abstract
Deep reinforcement learning (RL) has recently been successfully applied to networking contexts including routing, flow scheduling, congestion control, packet classification, cloud resource management, and video streaming. Deep-RL-driven systems automate decision making, and have been shown to outperform state-of-the-art handcrafted systems in important domains. However, the (typical) non-explainability of decisions induced by the deep learning machinery employed by these systems renders reasoning about crucial system properties, including correctness and security, extremely difficult. We show that despite the obscurity of decision making in these contexts, verifying that deep-RL-driven systems adhere to desired, designer-specified behavior, is achievable. To this end, we initiate the study of formal verification of deep RL and present Verily, a system for verifying deep-RL-based systems that leverages recent advances in verification of deep neural networks. We employ Verily to verify recently-introduced deep-RL-driven systems for adaptive video streaming, cloud resource management, and Internet congestion control. Our results expose scenarios in which deep-RL-driven decision making yields undesirable behavior. We discuss guidelines for building deep-RL-driven systems that are both safer and easier to verify.
Year
DOI
Venue
2019
10.1145/3341216.3342218
NetAI@SIGCOMM
Keywords
DocType
ISBN
congestion control, neural networks, reinforcement learning, resource management, verification, video streaming
Conference
978-1-4503-6872-8
Citations 
PageRank 
References 
3
0.40
0
Authors
4
Name
Order
Citations
PageRank
Yafim Kazak130.40
Clark Barrett21268108.65
Guy Katz326117.17
Michael Schapira4112279.89