Title
A Methodology to Efficiently Compare Operating System Stability
Abstract
Despite decades of advances in software engineering, operating systems (OSes) are still plagued by crashes due to software faults, calling for techniques to improve OS stability when faults occur. Evaluating such techniques requires a way to compare the stability of different OSes that is both representative of real faults and scales to the large code bases of modern OSes and a large (and statistically sound) number of experiments. In this paper, we propose a widely applicable methodology meeting all such requirements. Our methodology relies on a novel fault injection strategy based on a combination of static and run-time instrumentation, which yields representative software faults while drastically reducing the instrumentation time and thus greatly enhancing scalability. To guarantee unbiased and comparable results, finally, our methodology relies on the use of pre- and post tests to isolate the direct impact of faults from the stability of the OS itself. We demonstrate our methodology by comparing the stability of Linux and MINIX 3, saving a total of 115 computer-days for the 12,000 Linux fault injection runs compared to the traditional approach of re-instrumenting for every run.
Year
DOI
Venue
2015
10.1109/HASE.2015.22
HASE
Keywords
Field
DocType
operating systems,scalability,robustness,testing,linux
Computer science,Software fault tolerance,Robustness (computer science),Real-time computing,Software,Fault injection,Reliability engineering,Operating system,Embedded system,Scalability
Conference
ISSN
Citations 
PageRank 
1530-2059
1
0.35
References 
Authors
19
4
Name
Order
Citations
PageRank
Erik van der Kouwe1589.55
Cristiano Giuffrida287649.61
Razvan Ghituletez310.35
Andrew S. Tanenbaum43803745.87