Paper Info
Title
CloudRaid: hunting concurrency bugs in the cloud via log-mining.
Abstract
Cloud systems suffer from distributed concurrency bugs, which are notoriously difficult to detect and often lead to data loss and service outage. This paper presents CloudRaid, a new effective tool to battle distributed concurrency bugs. CloudRaid automatically detects concurrency bugs in cloud systems, by analyzing and testing those message orderings that are likely to expose errors. We observe that large-scale online cloud applications process millions of user requests per second, exercising many permutations of message orderings extensively. Those already sufficiently-tested message orderings are unlikely to expose errors. Hence, CloudRaid mines logs from previous executions to uncover those message orderings which are feasible, but not sufficiently tested. Specifically, CloudRaid tries to flip the order of a pair of messages <S,P> if they may happen in parallel, but S always arrives before P from existing logs, i.e., excercising the order P ↣ S. The log-based approach makes it suitable to live systems. We have applied CloudRaid to automatically test four representative distributed systems: Apache Hadoop2/Yarn, HBase, HDFS and Cassandra. CloudRaid can automatically test 40 different versions of the 4 systems (10 versions per system) in 35 hours, and can successfully trigger 28 concurrency bugs, including 8 new bugs that have never been found before. The 8 new bugs have all been confirmed by their original developers, and 3 of them are considered as critical bugs that have already been fixed.
Year
DOI
Venue
2018
10.1145/3236024.3236071
ESEC/SIGSOFT FSE
Keywords
Field
DocType
Distributed Systems,Concurrency Bugs,Bug Detection,Cloud Computing
Yarn,Cloud systems,Data loss,Concurrency,Computer science,Permutation,Theoretical computer science,Log mining,Distributed computing,Cloud computing
Conference
ISBN
Citations 
PageRank 
978-1-4503-5573-5
4
0.41
References 
Authors
0
4
Name
Order
Citations
PageRank
Jie Lu1235.89
Feng Li2112.59
Lian Li318940.80
Xiaobing Feng4906112.55