Paper Info
Title
SupMR: Circumventing Disk and Memory Bandwidth Bottlenecks for Scale-up MapReduce
Abstract
Reading input from primary storage (i.e. the ingest phase) and aggregating results (i.e. the merge phase) are important pre- and post-processing steps in large batch computations. Unfortunately, today's data sets are so large that the ingest and merge job phases are now performance bottlenecks. In this paper, we mitigate the ingest and merge bottlenecks by leveraging the scale-up MapReduce model. We introduce an ingest chunk pipeline and a merge optimization that increases CPU utilization (50 - 100%) and job phase speedups (1.16× - 3.13×) for the ingest and merge phases. Our techniques are based on well-known algorithms and scale-out MapReduce optimizations, but applying them to a scale-up computation framework to mitigate the ingest and merge bottlenecks is novel.
Year
DOI
Venue
2014
10.1109/IPDPSW.2014.168
Parallel & Distributed Processing Symposium Workshops
Keywords
Field
DocType
applications, architectures, distributed systems, distributed applications, performance measurements,pipelines,instruction sets,parallel processing,merging,distributed applications,applications,computational modeling,distributed systems,data handling
Data set,Memory bandwidth,SCALE-UP,Computer science,CPU time,Parallel computing,Merge (version control),Distributed computing,Computation
Conference
Citations 
PageRank 
References 
3
0.39
12
Authors
5
Name
Order
Citations
PageRank
Michael Sevilla1111.63
Ike Nassi217948.46
Kleoni Ioannidou31076.70
Scott A. Brandt4166394.81
Carlos Maltzahn5120187.49