Paper Info
Title
Automated oracles: an empirical study on cost and effectiveness
Abstract
Software testing is an effective, yet expensive, method to improve software quality. Test automation, a potential way to reduce testing cost, has received enormous research attention recently, but the so-called “oracle problem” (how to decide the PASS/FAIL outcome of a test execution) is still a major obstacle to such cost reduction. We have extensively investigated state-of-the-art works that contribute to address this problem, from areas such as specification mining and model inference. In this paper, we compare three types of automated oracles: Data invariants, Temporal invariants, and Finite State Automata. More specifically, we study the training cost and the false positive rate; we evaluate also their fault detection capability. Seven medium to large, industrial application subjects and real faults have been used in our empirical investigation.
Year
DOI
Venue
2013
10.1145/2491411.2491434
ESEC / SIGSOFT FSE
Keywords
Field
DocType
training cost,cost reduction,temporal invariants,test execution,test automation,software testing,data invariants,fail outcome,empirical study,software quality,oracle problem,automated oracle
False positive rate,Computer science,Fault detection and isolation,Oracle,Finite-state machine,Automation,Software quality,Cost reduction,Reliability engineering,Empirical research
Conference
Citations 
PageRank 
References 
16
0.68
13
Authors
3
Name
Order
Citations
PageRank
Cu D. Nguyen122414.19
Alessandro Marchetto257338.69
Paolo Tonella33559224.88