Paper Info
Title
Efficient Prototyping of Fault Tolerant Map-Reduce Applications with Docker-Hadoop
Abstract
Prototyping and testing distributed systems is considered to be a hard task because it is not always possible to reproduce a given sequence of events. While simulations may help on this task, they cannot replace test and validation with real systems. In this paper we present Docker-Hadoop, a container-based virtualization platform designed to prototype, test and deploy MapReduce applications and systems. This tool allowed us to test and reproduce fault-tolerance scenarios that are especially interesting in the context of the PER-MARE project, which aims at adapting the Hadoop framework to the case pervasive systems. Indeed, we developed a fault-tolerant component that can circumvent the limitations from original Hadoop and prevent the job scheduling stall in the case of failures or network disconnections. Thanks to Docker-Hadoop, we could easily prototype and test our improved Hadoop, with the first scalability and speedup results being presented in this paper.
Year
DOI
Venue
2015
10.1109/IC2E.2015.73
IC2E
Keywords
Field
DocType
fault tolerance,virtualisation,virtualization,prototyping,scalability,kernel,ubiquitous computing,parallel programming
Virtualization,Pervasive systems,Computer science,Fault tolerance,Job scheduler,Real systems,Speedup,Embedded system,Scalability
Conference
Citations 
PageRank 
References 
0
0.34
0
Authors
4
Name
Order
Citations
PageRank
Javier Rey150.86
Matias Cogorno250.86
Sergio Nesmachnow347248.10
Luiz Angelo Steffenel46213.84