Title
Testing the untestable: model testing of complex software-intensive systems.
Abstract
Increasingly, we are faced with systems that are untestable, meaning that traditional testing methods are expensive, time-consuming or infeasible to apply due to factors such as the systems' continuous interactions with the environment and the deep intertwining of software with hardware. In this paper we outline our vision to enable testing of untestable systems. Our key idea is to frame testing on models rather than operational systems. We refer to such testing as model testing. Our goal is to raise the level of abstraction of testing from operational systems to models of their behaviors and properties. The models that underlie model testing are executable representations of the relevant aspects of a system and its environment, alongside the risks of system failures. Such models necessarily have uncertainties due to complex, dynamic environment behaviors and the unknowns about the system. This makes it crucial for model testing to be uncertainty-aware. We propose to synergistically combine metaheuristic search, increasingly used in traditional software testing, with system and risk models to drive the search for faults that entail the most risk. We expect model testing to bring early and cost-effective automation to the testing of many critical systems that defy existing automation techniques, thus significantly improving the dependability of such systems.
Year
DOI
Venue
2016
10.1145/2889160.2889212
ICSE (Companion Volume)
Field
DocType
Citations 
System integration testing,Systems engineering,Computer science,System testing,Real-time computing,Software performance testing,White-box testing,Software reliability testing,Model-based testing,Acceptance testing,Test strategy
Conference
14
PageRank 
References 
Authors
0.63
8
4
Name
Order
Citations
PageRank
Lionel C. Briand18795481.98
Shiva Nejati283047.57
Mehrdad Sabetzadeh398861.84
Domenico Bianculli438129.75