Title
The impact of fault models on software robustness evaluations
Abstract
Following the design and in-lab testing of software, the evaluation of its resilience to actual operational perturbations in the field is a key validation need. Software-implemented fault injection (SWIFI) is a widely used approach for evaluating the robustness of software components. Recent research [24, 18] indicates that the selection of the applied fault model has considerable influence on the results of SWIFI-based evaluations, thereby raising the question how to select appropriate fault models (i.e. that provide justified robustness evidence). This paper proposes several metrics for comparatively evaluating fault models's abilities to reveal robustness vulnerabilities. It demonstrates their application in the context of OS device drivers by investigating the influence (and relative utility) of four commonly used fault models, i.e. bit flips (in function parameters and in binaries), data type dependent parameter corruptions, and parameter fuzzing. We assess the efficiency of these models at detecting robustness vulnerabilities during the SWIFI evaluation of a real embedded operating system kernel and discuss application guidelines for our metrics alongside.
Year
DOI
Venue
2011
10.1145/1985793.1985801
ICSE
Keywords
Field
DocType
software-implemented fault injection,appropriate fault model,application guideline,applied fault model,software robustness evaluation,swifi-based evaluation,robustness evidence,fault model,swifi evaluation,robustness vulnerability,considerable influence,software complexity,measurement,robustness,servers,operating system,software fault tolerance,software metrics,software component,robustness testing
Fuzz testing,Robustness testing,Computer science,Software fault tolerance,Real-time computing,Robustness (computer science),Software metric,Component-based software engineering,Reliability engineering,Fault model,Fault injection
Conference
Citations 
PageRank 
References 
7
0.50
18
Authors
4
Name
Order
Citations
PageRank
Stefan Winter1689.04
Constantin Sârbu2201.77
Neeraj Suri31040112.91
Brendan Murphy4119345.91