Name
Abstract | ||
|---|---|---|
An ensemble is a group of learning models that jointly solve a problem. However, the ensembles generated by existing techniques are sometimes unnecessarily large, which can lead to extra memory usage, computational costs, and occasional decreases in effectiveness. The purpose of ensemble pruning is to search for a good subset of ensemble members that performs as well as, or better than, the original ensemble. This subset selection problem is a combinatorial optimization problem and thus finding the exact optimal solution is computationally prohibitive. Various heuristic methods have been developed to obtain an approximate solution. However, most of the existing heuristics use simple greedy search as the optimization method, which lacks either theoretical or empirical quality guarantees. In this paper, the ensemble subset selection problem is formulated as a quadratic integer programming problem. By applying semi-definite programming (SDP) as a solution technique, we are able to get better approximate solutions. Computational experiments show that this SDP-based pruning algorithm outperforms other heuristics in the literature. Its application in a classifier-sharing study also demonstrates the effectiveness of the method. |
Year | Venue | Keywords |
|---|---|---|
2006 | Journal of Machine Learning Research | combinatorial optimization problem,ensemble pruning,semi-definite programming,ensemble pruning via semi-definite,ensemble member,quadratic integer programming problem,good subset,knowledge sharing,subset selection problem,approximate solution,exact optimal solution,heuristics,ensemble subset selection problem,original ensemble,computer experiment |
Field | DocType | Volume |
Subset sum problem,Heuristic,Mathematical optimization,Greedy algorithm,Heuristics,Artificial intelligence,Approximate solution,Ensemble learning,Machine learning,Mathematics,Semidefinite programming,Pruning | Journal | 7, |
ISSN | Citations | PageRank |
1532-4435 | 136 | 3.82 |
References | Authors | |
29 | 3 | |
Name | Order | Citations | PageRank |
|---|---|---|---|
| Yi Zhang | 1 | 214 | 10.52 |
| Samuel Burer | 2 | 1148 | 73.09 |
| W. Nick Street | 3 | 1828 | 155.26 |