Paper Info
Title
lpt: A Tool for Tuning the Level of Parallelism of Spark Applications
Abstract
Spark is increasingly becoming the platform of choice for several big-data analyses mainly due to its fast, fault-tolerant, and in-memory processing model. Despite the popularity and maturity of the Spark framework, tuning Spark applications to achieve high performance remains challenging. In this paper, we present lpt, a novel tool that assists users in improving the level of parallelism of applications running on top of Spark in the local mode. lpt helps users tune the level of parallelism of Spark applications to spawn a number of tasks able to fully exploit the available computing resources. Our evaluation results show that optimizations guided by lpt can achieve speedups up to 2.72x.
Year
DOI
Venue
2018
10.1109/APSEC.2018.00080
2018 25th Asia-Pacific Software Engineering Conference (APSEC)
Keywords
Field
DocType
Sparks,Task analysis,Parallel processing,Tuning,Measurement,Instruments,Servers
Spark (mathematics),Computer science,Real-time computing,Computer engineering
Conference
ISSN
ISBN
Citations 
1530-1362
978-1-7281-1970-0
0
PageRank 
References 
Authors
0.34
0
3
Name
Order
Citations
PageRank
Eduardo Rosales122.06
Andrea Rosà26312.04
Walter Binder3107792.58