Name
Abstract | ||
|---|---|---|
In the supervised classification, large training data are very common, and decision trees are widely used. However, as some bottlenecks such as memory restrictions, time complexity, or data complexity, many supervised classifiers including classical C4.5 tree cannot directly handle big data. One solution for this problem is to design a highly parallelized learning algorithm. Motivated by this, we propose a parallelized C4.5 decision tree algorithm based on MapReduce (MR-C4.5-Tree) with 2 parallelized methods to build the tree nodes. First, an information entropy-based parallelized attribute selection method (MR-A-S) on several subsets for MR-C4.5-Tree is proposed to confirm the best splitting attribute and the cut points. Then, a data splitting method (MR-D-S) in parallel is presented to partition the training data into subsets. At last, we introduce the MR-C4.5-Tree learning algorithm that grows in a top-down recursive way. Besides, the depth of the constructed decision tree, the number of samples and the maximal class probability in each tree node are used as the termination conditions to avoid the over-partitioning problem. Experimental studies show the feasibility and the good performance of the proposed parallelized MR-C4.5-Tree algorithm. |
Year | DOI | Venue |
|---|---|---|
2017 | 10.1002/cpe.4015 | CONCURRENCY AND COMPUTATION-PRACTICE & EXPERIENCE |
Keywords | Field | DocType |
C4.5,decision trees,MapReduce,parallel computing | Decision tree,Feature selection,Computer science,Parallel computing,Time complexity,ID3 algorithm,Entropy (information theory),Big data,Decision tree learning,Distributed computing,Incremental decision tree | Journal |
Volume | Issue | ISSN |
29 | 8 | 1532-0626 |
Citations | PageRank | References |
3 | 0.41 | 9 |
Authors | ||
4 | ||
Name | Order | Citations | PageRank |
|---|---|---|---|
| Yashuang Mu | 1 | 3 | 0.75 |
| Xiaodong Liu | 2 | 492 | 28.50 |
| Zhihao Yang | 3 | 73 | 15.35 |
| Xiaolin Liu | 4 | 3 | 0.41 |