Title
On Measuring and Quantifying Performance: Error Rates, Surrogate Loss, and an Example in SSL.
Abstract
In various approaches to learning, notably in domain adaptation, active learning, learning under covariate shift, semi-supervised learning, learning with concept drift, and the like, one often wants to compare a baseline classifier to one or more advanced (or at least different) strategies. In this chapter, we basically argue that if such classifiers, in their respective training phases, optimize a so-called surrogate loss that it may also be valuable to compare the behavior of this loss on the test set, next to the regular classification error rate. It can provide us with an additional view on the classifiers' relative performances that error rates cannot capture. As an example, limited but convincing empirical results demonstrates that we may be able to find semi-supervised learning strategies that can guarantee performance improvements with increasing numbers of unlabeled data in terms of log-likelihood. In contrast, the latter may be impossible to guarantee for the classification error rate.
Year
Venue
DocType
2017
CoRR
Journal
Volume
ISSN
Citations 
abs/1707.04025
In Handbook of Pattern Recognition and Computer Vision (pp. 53-68) (2016)
0
PageRank 
References 
Authors
0.34
0
3
Name
Order
Citations
PageRank
Marco Loog11796154.31
Jesse H. Krijthe200.34
Are Charles Jensen300.34