Paper Info
Title
SSS: An Implementation of Key-Value Store Based MapReduce Framework
Abstract
MapReduce has been very successful in implementing large-scale data-intensive applications. Because of its simple programming model, MapReduce has also begun being utilized as a programming tool for more general distributed and parallel applications, e.g., HPC applications. However, its applicability is limited due to relatively inefficient runtime performance and hence insufficient support for flexible workflow. In particular, the performance problem is not negligible in iterative MapReduce applications. On the other hand, today, HPC community is going to be able to utilize very fast and energy-efficient Solid State Drives (SSDs) with 10 Gbit/sec-class read/write performance. This fact leads us to the possibility to develop ``High-Performance MapReduce'', so called. From this perspective, we have been developing a new MapReduce framework called ``SSS'' based on distributed key-value store (KVS). In this paper, we first discuss the limitations of existing MapReduce implementations and present the design and implementation of SSS. Although our implementation of SSS is still in a prototype stage, we conduct two benchmarks for comparing the performance of SSS and Hadoop. The results indicate that SSS performs 1-10 times faster than Hadoop.
Year
DOI
Venue
2010
10.1109/CloudCom.2010.89
CloudCom
Keywords
Field
DocType
mapreduce framework,performance problem,mapreduce implementation,key-value store,simple programming model,hpc application,iterative mapreduce application,high-performance mapreduce,hpc community,new mapreduce framework,programming tool,inefficient runtime performance,instruction sets,programming,key value store,distributed databases,distributed processing,servers,prototypes,programming model,energy efficient,data models
Data modeling,Programming paradigm,Instruction set,Computer science,Server,Implementation,Associative array,Distributed database,Workflow,Distributed computing
Conference
Citations 
PageRank 
References 
9
0.90
11
Authors
4
Name
Order
Citations
PageRank
Hirotaka Ogawa119623.58
Hidemoto Nakada2956118.87
Ryousei Takano35116.12
Tomohiro Kudoh434450.92