Paper Info
Title
Optimized Common Parameter Set Extraction Framework By Multiple Benchmarking Applications On A Big Data Platform
Abstract
This research proposes the methodology to extract common configuration parameter set by applying multiple benchmarking applications include TeraSort, TestDFSIO, and MrBench on the Hadoop distributed file system. The parameter search space conceptually conducted named Omega(x) to hold status of all parameter values and its evaluation results for every stage to eventually reduce benchmarking cost. In the process of determining parameter set for each stage, one parameter and its associated values selected which is reduced system performance in terms of overall execution time difference that are measured by multiple applications on a Hadoop cluster. The experimental results demonstrate the proposed extended greedy manner provide a feasible benchmark model for the multiple MapReduce tasks. This model classified several candidate parameter value sets that can be reduced the overall execution time by 27% of the values against Hadoop default settings. Moreover, we propose e-heuristic greedy with alternative parameter selection model to evaluate second candidate parameter value which will lead global optimum by returning back to the previous stage if local minimum is not found at the current stage compare to the previous ones. (C) 2018 The Authors. Published by Atlantis Press SARL.
Year
DOI
Venue
2018
10.2991/ijndc.2018.6.4.1
INTERNATIONAL JOURNAL OF NETWORKED AND DISTRIBUTED COMPUTING
Keywords
Field
DocType
Big data, Hadoop, configuration, performance tuning
Data mining,Computer science,Big data,Performance tuning,Benchmarking
Journal
Volume
Issue
ISSN
6
4
2211-7938
Citations 
PageRank 
References 
0
0.34
0
Authors
5
Name
Order
Citations
PageRank
Jongyeop Kim121.73
Abhilash Kancharla200.68
Jongho Seol300.68
Noh-Jin Park400.34
Nohpill Park57217.90