Title
Specifying And Evaluating Quality Metrics For Vision-Based Perception Systems
Abstract
Robust perception algorithms are a vital ingredient for autonomous systems such as self-driving vehicles. Checking the correctness of perception algorithms such as those based on deep convolutional neural networks (CNN) is a formidable challenge problem. In this paper, we suggest the use of Timed Quality Temporal Logic (TQTL) as a formal language to express desirable spatio-temporal properties of a perception algorithm processing a video. While perception algorithms are traditionally tested by comparing their performance to ground truth labels, we show how TQTL can be a useful tool to determine quality of perception, and offers an alternative metric that can give useful information, even in the absence of ground truth labels. We demonstrate TQTL monitoring on two popular CNNs: YOLO and SqueezeDet, and give a comparative study of the results obtained for each architecture.
Year
DOI
Venue
2019
10.23919/DATE.2019.8715114
2019 DESIGN, AUTOMATION & TEST IN EUROPE CONFERENCE & EXHIBITION (DATE)
Keywords
Field
DocType
Temporal Logic, Monitoring, Autonomous vehicles, Perception, Image processing, Quality Metrics
Computer science,Real-time computing,Vision based,Human–computer interaction,Perception
Conference
ISSN
Citations 
PageRank 
1530-1591
0
0.34
References 
Authors
0
7
Name
Order
Citations
PageRank
Anand Balakrishnan112.05
Aniruddh G. Puranic200.34
Xin Qin312.06
Adel Dokhanchi4255.90
Jyotirmoy V. Deshmukh531729.18
Heni Ben Amor635935.77
Georgios E. Fainekos780452.65