Title
Cost-Aware Stage-Based Experimentation: Challenges and Emerging Results
Abstract
Experimentation at post-deployment phases (in production environments) can be a powerful tool for both learning how a deployed system operates and how it is being used. Though this knowledge is invaluable for optimization of the system, collecting it may require long time and experiments may even worsen the system with negative effects on users and business. This calls for methods for performing experimentation in production environments that balance the profit of experimentation with its cost. In this paper, we describe related challenges and our emerging results towards cost-aware stage-based experimentation. In particular, we aim for performing experiments that optimize towards their profit while making sure that the overall experimentation cost (e.g. total experimentation time) stays within given bounds. First, we illustrate the challenges and needs of such experimentation in two use cases from different domains. Second, we describe the main concepts behind our method in a semi-formal notation. Third, we exemplify the method by applying it in the two use cases and we report interesting first results.
Year
DOI
Venue
2018
10.1109/ICSA-C.2018.00027
2018 IEEE International Conference on Software Architecture Companion (ICSA-C)
Keywords
Field
DocType
experimentation,cost-aware,optimization
Notation,Use case,Software engineering,Computer science,Benchmark (computing)
Conference
ISBN
Citations 
PageRank 
978-1-5386-6586-2
2
0.40
References 
Authors
10
6
Name
Order
Citations
PageRank
Ilias Gerostathopoulos125426.55
Christian Prehofer293877.43
Lubomír Bulej316520.20
Tomás Bures418728.93
Vojtech Horký520.40
Petr Tuma627631.77