Title | ||
---|---|---|
Optimized Common Parameter Set Extraction Framework By Multiple Benchmarking Applications On A Big Data Platform |
Abstract | ||
---|---|---|
This research proposes the methodology to extract common configuration parameter set by applying multiple benchmarking applications include TeraSort, TestDFSIO, and MrBench on the Hadoop distributed file system. The parameter search space conceptually conducted named Omega(x) to hold status of all parameter values and its evaluation results for every stage to eventually reduce benchmarking cost. In the process of determining parameter set for each stage, one parameter and its associated values selected which is reduced system performance in terms of overall execution time difference that are measured by multiple applications on a Hadoop cluster. The experimental results demonstrate the proposed extended greedy manner provide a feasible benchmark model for the multiple MapReduce tasks. This model classified several candidate parameter value sets that can be reduced the overall execution time by 27% of the values against Hadoop default settings. Moreover, we propose e-heuristic greedy with alternative parameter selection model to evaluate second candidate parameter value which will lead global optimum by returning back to the previous stage if local minimum is not found at the current stage compare to the previous ones. (C) 2018 The Authors. Published by Atlantis Press SARL. |
Year | DOI | Venue |
---|---|---|
2018 | 10.2991/ijndc.2018.6.4.1 | INTERNATIONAL JOURNAL OF NETWORKED AND DISTRIBUTED COMPUTING |
Keywords | Field | DocType |
Big data, Hadoop, configuration, performance tuning | Data mining,Computer science,Big data,Performance tuning,Benchmarking | Journal |
Volume | Issue | ISSN |
6 | 4 | 2211-7938 |
Citations | PageRank | References |
0 | 0.34 | 0 |
Authors | ||
5 |
Name | Order | Citations | PageRank |
---|---|---|---|
Jongyeop Kim | 1 | 2 | 1.73 |
Abhilash Kancharla | 2 | 0 | 0.68 |
Jongho Seol | 3 | 0 | 0.68 |
Noh-Jin Park | 4 | 0 | 0.34 |
Nohpill Park | 5 | 72 | 17.90 |