Title
Selection of decision stumps in bagging ensembles
Abstract
This article presents a comprehensive study of different ensemble pruning techniques applied to a bagging ensemble composed of decision stumps. Six different ensemble pruning methods are tested. Four of these are greedy strategies based on first reordering the elements of the ensemble according to some rule that takes into account the complementarity of the predictors with respect to the classification task. Subensembles of increasing size are then constructed by incorporating the ordered classifiers one by one. A halting criterion stops the aggregation process before the complete original ensemble is recovered. The other two approaches are selection techniques that attempt to identify optimal subensembles using either genetic algorithms or semidefinite programming. Experiments performed on 24 benchmark classification tasks show that the selection of a small subset (≅ 10-15%) of the original pool of stumps generated with bagging can significantly increase the accuracy and reduce the complexity of the ensemble.
Year
DOI
Venue
2007
10.1007/978-3-540-74690-4_33
ICANN (1)
Keywords
Field
DocType
selection technique,different ensemble pruning technique,classification task,original pool,comprehensive study,benchmark classification task,different ensemble pruning method,aggregation process,complete original ensemble,bagging ensemble,decision stump,genetic algorithm
Complementarity (molecular biology),Pattern recognition,Computer science,Generalization error,Artificial intelligence,Ensemble learning,Machine learning,Semidefinite programming,Genetic algorithm,Pruning
Conference
Volume
ISSN
ISBN
4668
0302-9743
3-540-74689-7
Citations 
PageRank 
References 
6
0.49
9
Authors
3
Name
Order
Citations
PageRank
Gonzalo Martínez-Muñoz152423.76
Daniel Hernández-Lobato244026.10
Alberto Suárez348722.33