Paper Info
Title
Prioritizing manual test cases in rapid release environments.
Abstract
Test case prioritization is an important testing activity, in practice, specially for large scale systems. The goal is to rank the existing test cases in a way that they detect faults as soon as possible, so that any partial execution of the test suite detects the maximum number of defects for the given budget. Test prioritization becomes even more important when the test execution is time consuming, for example, manual system tests versus automated unit tests. Most existing test case prioritization techniques are based on code coverage, which requires access to source code. However, manual testing is mainly performed in a black-box manner (manual testers do not have access to the source code). Therefore, in this paper, the existing test case prioritization techniques (e.g. diversity-based and history-based techniques) are examined and modified to be applicable on manual black-box system testing. An empirical study on four older releases of desktop Firefox showed that none of the techniques were strongly dominating the others in all releases. However, when nine more recent releases of desktop Firefox, where the development has been moved from a traditional to a more agile and rapid release environment, were studied, a very significant difference between the history-based approach and its alternatives was observed. The higher effectiveness of the history-based approach compared with alternatives also held on 28 additional rapid releases of other Firefox projects - mobile Firefox and tablet Firefox. The conclusion of the paper is that test cases in rapid release environments can be very effectively prioritized for execution, based on their historical failure knowledge. In particular, it is the recency of historical knowledge that explains its effectiveness in rapid release environments rather than other changes in the process. Copyright (C) 2016 John Wiley & Sons, Ltd.
Year
DOI
Venue
2017
10.1002/stvr.1609
SOFTWARE TESTING VERIFICATION & RELIABILITY
Keywords
Field
DocType
test case prioritization,rapid release,manual testing,text mining,historical data
Test suite,Code coverage,Computer science,System testing,Source code,Manual testing,Unit testing,Agile software development,Test case,Reliability engineering,Database
Journal
Volume
Issue
ISSN
27.0
SP6
0960-0833
Citations 
PageRank 
References 
8
0.47
23
Authors
4
Name
Order
Citations
PageRank
Hadi Hemmati162227.54
Zhihan Fang280.47
Mika V. Mäntylä339327.44
Bram Adams483855.92