Paper Info
Title
Ensuring the Observability of Structural Test Obligations
Abstract
Test adequacy criteria are widely used to guide test creation. However, many of these criteria are sensitive to statement structure or the choice of test oracle. This is because such criteria ensure that execution reaches the element of interest, but impose no constraints on the execution path after this point. We are not guaranteed to observe a failure just because a fault is triggered. To address this issue, we have proposed the concept of observability-an extension to coverage criteria based on Boolean expressions that combines the obligations of a host criterion with an additional path condition that increases the likelihood that a fault encountered will propagate to a monitored variable. Our study, conducted over five industrial systems and an additional forty open-source systems, has revealed that adding observability tends to improve efficacy over satisfaction of the traditional criteria, with average improvements of 125.98 percent in mutation detection with the common output-only test oracle and per-model improvements of up to 1760.52 percent. Ultimately, there is merit to our hypothesis-observability reduces sensitivity to the choice of oracle and to the program structure.
Year
DOI
Venue
2020
10.1109/TSE.2018.2869146
IEEE Transactions on Software Engineering
Keywords
DocType
Volume
Software testing,automated test generation,test adequacy criteria,model-based test generation
Journal
46
Issue
ISSN
Citations 
7
0098-5589
0
PageRank 
References 
Authors
0.34
0
3
Name
Order
Citations
PageRank
Ying Meng100.34
Gregory Gay229915.27
Michael W. Whalen3109670.54