Paper Info
Title
CloudRank-D: benchmarking and ranking cloud computing systems for data processing applications.
Abstract
With the explosive growth of information, more and more organizations are deploying private cloud systems or renting public cloud systems to process big data. However, there is no existing benchmark suite for evaluating cloud performance on the whole system level. To the best of our knowledge, this paper proposes the first benchmark suite CloudRank-D to benchmark and rank cloud computing systems that are shared for running big data applications. We analyze the limitations of previous metrics, e. g., floating point operations, for evaluating a cloud computing system, and propose two simple metrics: data processed per second and data processed per Joule as two complementary metrics for evaluating cloud computing systems. We detail the design of CloudRank-D that considers representative applications, diversity of data characteristics, and dynamic behaviors of both applications and system software platforms. Through experiments, we demonstrate the advantages of our proposed metrics. In several case studies, we evaluate two small-scale deployments of cloud computing systems using CloudRank-D. © 2012 Higher Education Press and Springer-Verlag Berlin Heidelberg.
Year
DOI
Venue
2012
10.1007/s11704-012-2118-7
Frontiers of Computer Science
Keywords
Field
DocType
data center systems,clouds,big data applications,benchmarks,evaluation metrics
System software,Data processing,Suite,Ranking,Floating point,Computer science,Big data,Database,Benchmarking,Cloud computing
Journal
Volume
Issue
ISSN
6
4
16737466
Citations 
PageRank 
References 
42
2.01
18
Authors
8
Name
Order
Citations
PageRank
Chunjie Luo143421.86
Jianfeng Zhan276762.86
Zhen Jia333817.82
Lei Wang457746.85
Gang Lu531112.40
Lixin Zhang657145.96
Z. Chen73443271.62
SUN Ning-Hui8126897.37