Paper Info
Title
AveBoost2: Boosting for Noisy Data
Abstract
AdaBoost [4] is a well-known ensemble learning algorithm that constructs its base models in sequence. AdaBoost constructs a distribution over the training examples to create each base model. This distribution, represented as a vector, is constructed with the goal of making the next base model's mistakes uncorrelated with those of the previous base model [5]. We previously [71 developed an algorithm, AveBoost, that first constructed a distribution the same way as AdaBoost but then averaged it with the previous models' distributions to create the next base model's distribution. Our experiments demonstrated the superior accuracy of this approach. In this paper, we slightly revise our algorithm to obtain non-trivial theoretical results: bounds on the training error and generalization error (difference between training and test error). Our averaging process has a regularizing effect which leads us to a worse training error bound for our algorithm than for AdaBoost but a better generalization error bound. This leads us to suspect that our new algorithm works better than AdaBoost on noisy data. For this paper, we experimented with the data that we used in [71 both as originally supplied and with added label noise - some of the data has its original label changed randomly. Our algorithm's experimental performance improvement over AdaBoost is even greater on the noisy data than the original data.
Year
DOI
Venue
2004
10.1007/978-3-540-25966-4_3
Lecture Notes in Computer Science
Keywords
Field
DocType
ensemble learning,algorithms,information theory,generalization error,mathematical models,noise,machine learning,stability,orthogonal functions
Information theory,Signal processing,AdaBoost,Computer science,Signal-to-noise ratio,Algorithm,Boosting (machine learning),Ensemble learning,BrownBoost,Performance improvement
Conference
Volume
ISSN
Citations 
3077
0302-9743
18
PageRank 
References 
Authors
0.82
5
1
Name
Order
Citations
PageRank
nikunj c oza169454.32