Paper Info
Title
High-coverage metamorphic testing of concurrency support in C compilers
Abstract
We present a technique and automated toolbox for randomized testing of C compilers. Unlike prior compiler-testing approaches, we generate concurrent test cases in which threads communicate using fine-grained atomic operations, and we study actual compiler implementations rather than abstract mappings. Our approach is (1) to generate test cases with precise oracles directly from an axiomatization of the C concurrency model; (2) to apply metamorphic fuzzing to each test case, aiming to amplify the coverage they are likely to achieve on compiler codebases; and (3) to execute each fuzzed test case extensively on a range of real machines. Our tool, C4, benefits compiler developers in two ways. First, test cases generated by C4 can achieve line coverage of parts of the LLVM C compiler that are reached by neither the LLVM test suite nor an existing (sequential) C fuzzer. This information can be used to guide further development of the LLVM test suite and can also shed light on where and how concurrency-related compiler optimizations are implemented. Second, C4 can be used to gain confidence that a compiler implements concurrency correctly. As evidence of this, we show that C4 achieves high strong mutation coverage with respect to a set of concurrency-related mutants derived from a recent version of LLVM and that it can find historic concurrency-related bugs in GCC. As a by-product of concurrency-focused testing, C4 also revealed two previously unknown sequential compiler bugs in recent versions of GCC and the IBM XL compiler.
Year
DOI
Venue
2022
10.1002/stvr.1812
SOFTWARE TESTING VERIFICATION & RELIABILITY
Keywords
DocType
Volume
atomics, C11, concurrency, fuzz testing, program generation, weak memory
Journal
32
Issue
ISSN
Citations 
4
0960-0833
0
PageRank 
References 
Authors
0.34
0
3
Name
Order
Citations
PageRank
Matt Windsor100.34
Alastair F. Donaldson266152.35
John Wickerson314210.08