Paper Info
Title
Explaining Deep Convolutional Neural Networks via Latent Visual-Semantic Filter Attention
Abstract
Interpretability is an important property for visual mod-els as it helps researchers and users understand the in-ternal mechanism of a complex model. However, gener-ating semantic explanations about the learned representation is challenging without direct supervision to produce such explanations. We propose a general framework, La-tent Visual Semantic Explainer (LaViSE), to teach any ex-isting convolutional neural network to generate text de-scriptions about its own latent representations at the filter level. Our method constructs a mapping between the vi-sual and semantic spaces using generic image datasets, using images and category names. It then transfers the map-ping to the target domain which does not have semantic la-bels. The proposedframework employs a modular structure and enables to analyze any trained network whether or not its original training data is available. We show that our method can generate novel descriptions for learned filters beyond the set of categories defined in the training dataset and perform an extensive evaluation on multiple datasets. We also demonstrate a novel application of our method for unsupervised dataset bias analysis which allows us to auto-matically discover hidden biases in datasets or compare dif-ferent subsets without using additional labels. The dataset and code are made public to facilitate further research. 1 1 https://github.com/YuYang0901/LaViSE
Year
DOI
Venue
2022
10.1109/CVPR52688.2022.00815
IEEE Conference on Computer Vision and Pattern Recognition
Keywords
DocType
Volume
Explainable computer vision, Vision + language, Vision applications and systems
Conference
2022
Issue
Citations 
PageRank 
1
0
0.34
References 
Authors
0
3
Name
Order
Citations
PageRank
Yu Yang100.68
Seungbae Kim201.01
Jungseock Joo300.34