Paper Info
Title
Automated Verification of Load Tests Using Control Charts
Abstract
Load testing is an important phase in the software development process. It is very time consuming but there is usually little time for it. As a solution to the tight testing schedule, software companies automate their testing procedures. However, existing automation only reduces the time required to run load tests. The analysis of the test results is still performed manually. A typical load test outputs thousands of performance counters. Analyzing these counters manually requires time and tacit knowledge of the system-under-test from the performance engineers. The goal of this study is to derive an approach to automatically verify load tests' results. We propose an approach based on a statistical quality control technique called control charts. Our approach can a) automatically determine if a test run passes or fails and b) identify the subsystem where performance problem originated. We conduct two case studies on a large commercial telecommunication software and an open-source software system to evaluate our approach. Our results warrant further development of control chart based techniques in performance verification.
Year
DOI
Venue
2011
10.1109/APSEC.2011.59
APSEC
Keywords
Field
DocType
software company,load test,performance problem,control charts,open-source software system,performance verification,performance counter,automated verification,large commercial telecommunication software,performance engineer,software development process,load testing,performance testing,control chart,software quality,statistical quality control,system under test
Load testing,Computer science,Software system,Dynamic testing,Software performance testing,Real-time computing,Regression testing,Software development process,Statistical process control,Software quality,Reliability engineering
Conference
ISSN
ISBN
Citations 
1530-1362
978-1-4577-2199-1
10
PageRank 
References 
Authors
0.54
0
6
Name
Order
Citations
PageRank
Thanh H.D. Nguyen127011.84
Bram Adams279832.26
Zhen Ming Jiang378040.11
Ahmed E. Hassan45959287.68
Mohamed Nasser51074.57
Parminder Flora641619.50