Title
On the danger of coverage directed test case generation
Abstract
In the avionics domain, the use of structural coverage criteria is legally required in determining test suite adequacy. With the success of automated test generation tools, it is tempting to use these criteria as the basis for test generation. To more firmly establish the effectiveness of such approaches, we have generated and evaluated test suites to satisfy two coverage criteria using counterexample-based test generation and a random generation approach, contrasted against purely random test suites of equal size. Our results yield two key conclusions. First, coverage criteria satisfaction alone is a poor indication of test suite effectiveness. Second, the use of structural coverage as a supplement--not a target--for test generation can have a positive impact. These observations points to the dangers inherent in the increase in test automation in critical systems and the need for more research in how coverage criteria, generation approach, and system structure jointly influence test effectiveness.
Year
DOI
Venue
2012
10.1007/978-3-642-28872-2_28
FASE
Keywords
Field
DocType
test case generation,coverage criterion,test effectiveness,test suite adequacy,test automation,random test suite,automated test generation tool,test generation,test suite,test suite effectiveness,counterexample-based test generation
Test suite,Code coverage,System structure,Test Management Approach,Computer science,Avionics,Automation,Test case,Counterexample,Reliability engineering
Conference
Volume
ISSN
Citations 
7212
0302-9743
37
PageRank 
References 
Authors
1.10
26
4
Name
Order
Citations
PageRank
Matt Staats187132.29
Gregory Gay229915.27
Michael W. Whalen3109670.54
Mats Heimdahl4896.43