Paper Info
Title
Using controlled numbers of real faults and mutants to empirically evaluate coverage-based test case prioritization.
Abstract
Used to establish confidence in the correctness of evolving software, regression testing is an important, yet costly, task. Test case prioritization enables the rapid detection of faults during regression testing by reordering the test suite so that effective tests are run as early as is possible. However, a distinct lack of information about the regression faults found in complex real-world software forced prior experimental studies of these methods to use artificial faults called mutants. Using the Defects4J database of real faults, this paper presents the results of experiments evaluating the effectiveness of four representative test prioritization techniques. Since this paper's results show that prioritization is susceptible to high amounts of variance when only one fault is present, our experiments also control the number of real faults and mutants in the program subject to regression testing. Our overall findings are that, in comparison to mutants, real faults are harder for reordered test suites to quickly detect, suggesting that mutants are not a surrogate for real faults.
Year
DOI
Venue
2018
10.1145/3194733.3194735
AST@ICSE
Keywords
Field
DocType
test case prioritization,regression testing,real faults,defects4j
Test suite,Data mining,Regression,Computer science,Correctness,Oracle,Regression testing,White-box testing,Real-time computing,Test design,Software
Conference
ISSN
ISBN
Citations 
2377-8628
978-1-4503-5743-2
1
PageRank 
References 
Authors
0.35
28
4
Name
Order
Citations
PageRank
David Paterson181.19
Gregory M. Kapfhammer251535.21
Gordon Fraser32625116.22
phil mcminn4238497.58