Title
Stochastic Feature Mapping for PAC-Bayes Classification
Abstract
Hidden information derived from probabilistic generative models of data distributions can be used to construct features for discriminative classifiers. This observation has motivated the development of approaches that attempt to couple generative and discriminative models together for classification. However, existing approaches typically feed features derived from generative models to discriminative classifiers, and do not refine the generative models or the feature mapping functions based on classification results. In this paper, we propose a coupling mechanism developed under the PAC-Bayes framework that can fine-tune the generative models and the feature mapping functions iteratively to improve the classifier's performance. In our approach, a stochastic feature mapping, which is a function over the random variables of a generative model, is derived to generate feature vectors for a stochastic classifier. We construct a stochastic classifier over the feature mapping and derive the PAC-Bayes generalization bound for the classifier, for both supervised and semi-supervised learning. This allows us to jointly learn the feature mapping and the classifier by minimizing the bound with an EM-like iterative algorithm using labeled and unlabeled data. The resulting framework integrates the learning of the discriminative classifier and the generative model and allows iterative fine-tuning of the generative models, and the feedforward feature mappings based on task performance feedback. Our experiments show, in three distinct applications, this new framework produces a general classification tool with state-of-the-art performance.
Year
DOI
Venue
2012
10.1007/s10994-015-5525-9
Machine Learning
Keywords
Field
DocType
Stochastic feature mapping,PAC-Bayes generalization bound,Hybrid generative-discriminative classification
Feature vector,Generative topographic map,Pattern recognition,Iterative method,Artificial intelligence,Probabilistic logic,Linear classifier,Classifier (linguistics),Discriminative model,Machine learning,Mathematics,Generative model
Journal
Volume
Issue
ISSN
abs/1204.2609
1-3
0885-6125
Citations 
PageRank 
References 
5
0.44
28
Authors
4
Name
Order
Citations
PageRank
xiong li1688.63
Bin Wang21788246.68
Yuncai Liu31234185.16
Tai Sing Lee479488.73